Fold 0
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': None,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_0',
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': None,
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.374123
Best score so far is 0.374123 at epoch 0 
Epoch 1
Validation prc-auc = 0.370996
Epoch 2
Validation prc-auc = 0.360169
Epoch 3
Validation prc-auc = 0.351534
Epoch 4
Loss = 1.0880e-02, PNorm = 66.7751, GNorm = 5.3111, lr_0 = 6.5000e-04
Validation prc-auc = 0.368284
Epoch 5
Validation prc-auc = 0.349247
Epoch 6
Validation prc-auc = 0.346864
Epoch 7
Validation prc-auc = 0.362743
Epoch 8
Validation prc-auc = 0.371491
Epoch 9
Loss = 7.5740e-03, PNorm = 66.9166, GNorm = 2.9482, lr_0 = 9.9743e-04
Validation prc-auc = 0.361200
Epoch 10
Validation prc-auc = 0.438256
Best score so far is 0.438256 at epoch 10 
Epoch 11
Validation prc-auc = 0.333532
Epoch 12
Validation prc-auc = 0.292925
Epoch 13
Validation prc-auc = 0.328218
Epoch 14
Loss = 6.3431e-03, PNorm = 67.1015, GNorm = 3.1202, lr_0 = 9.8890e-04
Validation prc-auc = 0.337000
Epoch 15
Validation prc-auc = 0.345134
Epoch 16
Validation prc-auc = 0.377506
Epoch 17
Validation prc-auc = 0.401131
Epoch 18
Validation prc-auc = 0.423431
Epoch 19
Loss = 2.4696e-03, PNorm = 67.2554, GNorm = 1.2465, lr_0 = 9.8045e-04
Validation prc-auc = 0.447720
Best score so far is 0.447720 at epoch 19 
Epoch 20
Validation prc-auc = 0.458862
Best score so far is 0.458862 at epoch 20 
Epoch 21
Validation prc-auc = 0.448552
Epoch 22
Validation prc-auc = 0.432954
Epoch 23
Validation prc-auc = 0.373637
Epoch 24
Loss = 3.1616e-03, PNorm = 67.3754, GNorm = 1.7819, lr_0 = 9.7207e-04
Validation prc-auc = 0.342777
Epoch 25
Validation prc-auc = 0.340556
Epoch 26
Validation prc-auc = 0.377421
Epoch 27
Validation prc-auc = 0.429349
Epoch 28
Validation prc-auc = 0.430065
Epoch 29
Loss = 1.9449e-03, PNorm = 67.4864, GNorm = 0.6774, lr_0 = 9.6376e-04
Validation prc-auc = 0.426222
Epoch 30
Validation prc-auc = 0.424571
Epoch 31
Validation prc-auc = 0.423644
Epoch 32
Validation prc-auc = 0.409856
Epoch 33
Validation prc-auc = 0.377851
Epoch 34
Loss = 2.0139e-03, PNorm = 67.6007, GNorm = 1.2137, lr_0 = 9.5552e-04
Validation prc-auc = 0.382968
Epoch 35
Validation prc-auc = 0.401930
Epoch 36
Validation prc-auc = 0.400060
Epoch 37
Validation prc-auc = 0.368520
Epoch 38
Validation prc-auc = 0.367055
Epoch 39
Loss = 1.8782e-03, PNorm = 67.6889, GNorm = 1.9955, lr_0 = 9.4735e-04
Validation prc-auc = 0.363134
Epoch 40
Validation prc-auc = 0.373379
Epoch 41
Validation prc-auc = 0.395439
Epoch 42
Validation prc-auc = 0.424841
Epoch 43
Validation prc-auc = 0.427728
Epoch 44
Loss = 1.1462e-03, PNorm = 67.7826, GNorm = 0.7578, lr_0 = 9.3925e-04
Validation prc-auc = 0.396107
Epoch 45
Validation prc-auc = 0.394165
Epoch 46
Validation prc-auc = 0.389258
Epoch 47
Validation prc-auc = 0.389603
Epoch 48
Validation prc-auc = 0.413399
Epoch 49
Loss = 2.2099e-03, PNorm = 67.8696, GNorm = 1.4865, lr_0 = 9.3122e-04
Validation prc-auc = 0.437792
Epoch 50
Validation prc-auc = 0.422019
Epoch 51
Validation prc-auc = 0.406564
Epoch 52
Validation prc-auc = 0.386760
Epoch 53
Validation prc-auc = 0.384881
Epoch 54
Loss = 1.7921e-03, PNorm = 67.9461, GNorm = 0.7397, lr_0 = 9.2326e-04
Validation prc-auc = 0.399989
Epoch 55
Validation prc-auc = 0.403719
Epoch 56
Validation prc-auc = 0.410394
Epoch 57
Validation prc-auc = 0.445921
Epoch 58
Validation prc-auc = 0.482009
Best score so far is 0.482009 at epoch 58 
Epoch 59
Loss = 1.3921e-03, PNorm = 68.0176, GNorm = 2.2408, lr_0 = 9.1537e-04
Validation prc-auc = 0.480640
Epoch 60
Validation prc-auc = 0.497273
Best score so far is 0.497273 at epoch 60 
Epoch 61
Validation prc-auc = 0.491523
Epoch 62
Validation prc-auc = 0.484603
Epoch 63
Validation prc-auc = 0.459054
Epoch 64
Loss = 1.3181e-03, PNorm = 68.0988, GNorm = 3.3811, lr_0 = 9.0754e-04
Validation prc-auc = 0.435155
Epoch 65
Validation prc-auc = 0.391561
Epoch 66
Validation prc-auc = 0.401582
Epoch 67
Validation prc-auc = 0.414717
Epoch 68
Validation prc-auc = 0.393153
Epoch 69
Loss = 9.4981e-04, PNorm = 68.1918, GNorm = 1.1682, lr_0 = 8.9978e-04
Validation prc-auc = 0.372288
Epoch 70
Validation prc-auc = 0.379807
Epoch 71
Validation prc-auc = 0.384451
Epoch 72
Validation prc-auc = 0.367031
Epoch 73
Validation prc-auc = 0.350830
Epoch 74
Loss = 7.4398e-04, PNorm = 68.2876, GNorm = 0.6647, lr_0 = 8.9209e-04
Validation prc-auc = 0.345390
Epoch 75
Validation prc-auc = 0.363286
Epoch 76
Validation prc-auc = 0.401409
Epoch 77
Validation prc-auc = 0.447316
Epoch 78
Validation prc-auc = 0.466835
Epoch 79
Loss = 1.4710e-03, PNorm = 68.3640, GNorm = 2.7642, lr_0 = 8.8447e-04
Validation prc-auc = 0.453402
Epoch 80
Validation prc-auc = 0.413144
Epoch 81
Validation prc-auc = 0.365898
Epoch 82
Validation prc-auc = 0.361062
Epoch 83
Validation prc-auc = 0.348255
Epoch 84
Loss = 1.0233e-03, PNorm = 68.4738, GNorm = 0.9194, lr_0 = 8.7691e-04
Validation prc-auc = 0.375743
Epoch 85
Validation prc-auc = 0.361224
Epoch 86
Validation prc-auc = 0.282798
Epoch 87
Validation prc-auc = 0.277797
Epoch 88
Validation prc-auc = 0.365728
Epoch 89
Loss = 1.7050e-03, PNorm = 68.5809, GNorm = 0.9661, lr_0 = 8.6941e-04
Validation prc-auc = 0.412095
Epoch 90
Validation prc-auc = 0.417395
Epoch 91
Validation prc-auc = 0.381876
Epoch 92
Validation prc-auc = 0.339795
Epoch 93
Validation prc-auc = 0.275295
Epoch 94
Loss = 1.0776e-03, PNorm = 68.6966, GNorm = 1.6788, lr_0 = 8.6198e-04
Validation prc-auc = 0.196919
Epoch 95
Validation prc-auc = 0.222665
Epoch 96
Validation prc-auc = 0.235451
Epoch 97
Validation prc-auc = 0.271382
Epoch 98
Validation prc-auc = 0.262714
Epoch 99
Loss = 5.9422e-04, PNorm = 68.7989, GNorm = 1.0856, lr_0 = 8.5461e-04
Validation prc-auc = 0.269136
Epoch 100
Validation prc-auc = 0.319825
Epoch 101
Validation prc-auc = 0.355928
Epoch 102
Validation prc-auc = 0.368938
Epoch 103
Validation prc-auc = 0.372515
Epoch 104
Loss = 7.5453e-04, PNorm = 68.8786, GNorm = 1.0075, lr_0 = 8.4730e-04
Validation prc-auc = 0.363288
Epoch 105
Validation prc-auc = 0.273150
Epoch 106
Validation prc-auc = 0.254565
Epoch 107
Validation prc-auc = 0.246395
Epoch 108
Validation prc-auc = 0.273812
Epoch 109
Loss = 4.9766e-04, PNorm = 68.9468, GNorm = 0.9800, lr_0 = 8.4006e-04
Validation prc-auc = 0.365764
Epoch 110
Validation prc-auc = 0.377875
Epoch 111
Validation prc-auc = 0.378604
Epoch 112
Validation prc-auc = 0.380046
Epoch 113
Validation prc-auc = 0.358732
Epoch 114
Loss = 6.9759e-04, PNorm = 69.0156, GNorm = 0.3530, lr_0 = 8.3288e-04
Validation prc-auc = 0.357969
Epoch 115
Validation prc-auc = 0.360509
Epoch 116
Validation prc-auc = 0.367726
Epoch 117
Validation prc-auc = 0.383989
Epoch 118
Validation prc-auc = 0.388098
Epoch 119
Loss = 3.7117e-04, PNorm = 69.0753, GNorm = 0.1831, lr_0 = 8.2576e-04
Validation prc-auc = 0.378202
Epoch 120
Validation prc-auc = 0.373180
Epoch 121
Validation prc-auc = 0.368835
Epoch 122
Validation prc-auc = 0.367916
Epoch 123
Validation prc-auc = 0.383481
Epoch 124
Loss = 2.3820e-04, PNorm = 69.1394, GNorm = 0.9907, lr_0 = 8.1870e-04
Validation prc-auc = 0.389833
Epoch 125
Validation prc-auc = 0.415239
Epoch 126
Validation prc-auc = 0.422363
Epoch 127
Validation prc-auc = 0.424966
Epoch 128
Validation prc-auc = 0.441787
Epoch 129
Loss = 1.0973e-04, PNorm = 69.1994, GNorm = 0.0310, lr_0 = 8.1170e-04
Validation prc-auc = 0.429295
Epoch 130
Validation prc-auc = 0.414281
Epoch 131
Validation prc-auc = 0.391957
Epoch 132
Validation prc-auc = 0.362295
Epoch 133
Validation prc-auc = 0.368441
Epoch 134
Loss = 3.7160e-04, PNorm = 69.2493, GNorm = 0.8341, lr_0 = 8.0476e-04
Validation prc-auc = 0.387836
Epoch 135
Validation prc-auc = 0.402295
Epoch 136
Validation prc-auc = 0.415525
Epoch 137
Validation prc-auc = 0.419584
Epoch 138
Validation prc-auc = 0.422187
Epoch 139
Loss = 1.1383e-03, PNorm = 69.3193, GNorm = 3.0469, lr_0 = 7.9788e-04
Validation prc-auc = 0.409871
Epoch 140
Validation prc-auc = 0.383890
Epoch 141
Validation prc-auc = 0.339805
Epoch 142
Validation prc-auc = 0.349709
Epoch 143
Validation prc-auc = 0.387814
Epoch 144
Loss = 6.5890e-04, PNorm = 69.3942, GNorm = 1.3564, lr_0 = 7.9106e-04
Validation prc-auc = 0.389756
Epoch 145
Validation prc-auc = 0.419876
Epoch 146
Validation prc-auc = 0.446402
Epoch 147
Validation prc-auc = 0.470942
Epoch 148
Validation prc-auc = 0.457663
Epoch 149
Loss = 5.8775e-04, PNorm = 69.5130, GNorm = 1.8788, lr_0 = 7.8430e-04
Validation prc-auc = 0.451996
Epoch 150
Validation prc-auc = 0.442785
Epoch 151
Validation prc-auc = 0.427316
Epoch 152
Validation prc-auc = 0.414566
Epoch 153
Validation prc-auc = 0.400265
Epoch 154
Loss = 7.8229e-04, PNorm = 69.6124, GNorm = 1.7279, lr_0 = 7.7759e-04
Validation prc-auc = 0.405241
Epoch 155
Validation prc-auc = 0.408744
Epoch 156
Validation prc-auc = 0.408835
Epoch 157
Validation prc-auc = 0.417550
Epoch 158
Validation prc-auc = 0.433626
Epoch 159
Loss = 2.0514e-03, PNorm = 69.7042, GNorm = 2.2874, lr_0 = 7.7095e-04
Validation prc-auc = 0.448749
Epoch 160
Validation prc-auc = 0.463733
Epoch 161
Validation prc-auc = 0.464708
Epoch 162
Validation prc-auc = 0.469271
Epoch 163
Validation prc-auc = 0.457050
Epoch 164
Loss = 9.7847e-05, PNorm = 69.7576, GNorm = 0.0733, lr_0 = 7.6436e-04
Validation prc-auc = 0.447237
Epoch 165
Validation prc-auc = 0.408090
Epoch 166
Validation prc-auc = 0.372638
Epoch 167
Validation prc-auc = 0.369913
Epoch 168
Validation prc-auc = 0.403578
Epoch 169
Loss = 1.4771e-04, PNorm = 69.8010, GNorm = 0.1014, lr_0 = 7.5782e-04
Validation prc-auc = 0.414709
Epoch 170
Validation prc-auc = 0.428489
Epoch 171
Validation prc-auc = 0.427680
Epoch 172
Validation prc-auc = 0.432600
Epoch 173
Validation prc-auc = 0.433456
Epoch 174
Loss = 8.1660e-04, PNorm = 69.8304, GNorm = 0.2369, lr_0 = 7.5134e-04
Validation prc-auc = 0.420605
Epoch 175
Validation prc-auc = 0.397060
Epoch 176
Validation prc-auc = 0.398184
Epoch 177
Validation prc-auc = 0.407178
Epoch 178
Validation prc-auc = 0.404671
Epoch 179
Loss = 5.0783e-04, PNorm = 69.8637, GNorm = 1.6079, lr_0 = 7.4492e-04
Validation prc-auc = 0.410433
Epoch 180
Validation prc-auc = 0.403468
Epoch 181
Validation prc-auc = 0.411219
Epoch 182
Validation prc-auc = 0.434405
Epoch 183
Validation prc-auc = 0.443920
Epoch 184
Loss = 1.0807e-04, PNorm = 69.9056, GNorm = 0.5387, lr_0 = 7.3855e-04
Validation prc-auc = 0.428849
Epoch 185
Validation prc-auc = 0.425577
Epoch 186
Validation prc-auc = 0.389289
Epoch 187
Validation prc-auc = 0.387789
Epoch 188
Validation prc-auc = 0.396899
Epoch 189
Loss = 1.7708e-04, PNorm = 69.9393, GNorm = 0.8531, lr_0 = 7.3224e-04
Validation prc-auc = 0.390175
Epoch 190
Validation prc-auc = 0.373612
Epoch 191
Validation prc-auc = 0.365773
Epoch 192
Validation prc-auc = 0.360290
Epoch 193
Validation prc-auc = 0.376983
Epoch 194
Loss = 7.4826e-04, PNorm = 69.9707, GNorm = 0.2838, lr_0 = 7.2598e-04
Validation prc-auc = 0.372051
Epoch 195
Validation prc-auc = 0.362693
Epoch 196
Validation prc-auc = 0.384830
Epoch 197
Validation prc-auc = 0.419608
Epoch 198
Validation prc-auc = 0.422465
Epoch 199
Loss = 6.6759e-05, PNorm = 70.0233, GNorm = 0.0519, lr_0 = 7.1977e-04
Validation prc-auc = 0.439608
Epoch 200
Validation prc-auc = 0.455341
Epoch 201
Validation prc-auc = 0.459944
Epoch 202
Validation prc-auc = 0.451187
Epoch 203
Validation prc-auc = 0.443820
Epoch 204
Loss = 4.5548e-05, PNorm = 70.0563, GNorm = 0.1300, lr_0 = 7.1362e-04
Validation prc-auc = 0.443188
Epoch 205
Validation prc-auc = 0.437446
Epoch 206
Validation prc-auc = 0.428332
Epoch 207
Validation prc-auc = 0.424066
Epoch 208
Validation prc-auc = 0.424573
Epoch 209
Loss = 1.9223e-05, PNorm = 70.0785, GNorm = 0.0720, lr_0 = 7.0752e-04
Validation prc-auc = 0.424574
Epoch 210
Validation prc-auc = 0.407017
Epoch 211
Validation prc-auc = 0.387317
Epoch 212
Validation prc-auc = 0.398720
Epoch 213
Validation prc-auc = 0.378586
Epoch 214
Loss = 5.0191e-04, PNorm = 70.1113, GNorm = 0.0402, lr_0 = 7.0147e-04
Validation prc-auc = 0.369725
Epoch 215
Validation prc-auc = 0.354405
Epoch 216
Validation prc-auc = 0.349530
Epoch 217
Validation prc-auc = 0.343703
Epoch 218
Validation prc-auc = 0.348292
Epoch 219
Loss = 1.8108e-04, PNorm = 70.1554, GNorm = 0.0518, lr_0 = 6.9548e-04
Validation prc-auc = 0.346917
Epoch 220
Validation prc-auc = 0.357313
Epoch 221
Validation prc-auc = 0.360674
Epoch 222
Validation prc-auc = 0.366378
Epoch 223
Validation prc-auc = 0.374563
Epoch 224
Loss = 9.6685e-05, PNorm = 70.1905, GNorm = 0.3729, lr_0 = 6.8953e-04
Validation prc-auc = 0.371828
Epoch 225
Validation prc-auc = 0.361200
Epoch 226
Validation prc-auc = 0.352235
Epoch 227
Validation prc-auc = 0.348655
Epoch 228
Validation prc-auc = 0.344757
Epoch 229
Loss = 1.1164e-04, PNorm = 70.2218, GNorm = 0.4697, lr_0 = 6.8364e-04
Validation prc-auc = 0.338647
Epoch 230
Validation prc-auc = 0.333216
Epoch 231
Validation prc-auc = 0.332609
Epoch 232
Validation prc-auc = 0.334655
Epoch 233
Validation prc-auc = 0.335367
Epoch 234
Loss = 1.2579e-04, PNorm = 70.2599, GNorm = 0.6397, lr_0 = 6.7779e-04
Validation prc-auc = 0.345565
Epoch 235
Validation prc-auc = 0.339005
Epoch 236
Validation prc-auc = 0.338337
Epoch 237
Validation prc-auc = 0.344799
Epoch 238
Validation prc-auc = 0.353965
Epoch 239
Loss = 7.3734e-05, PNorm = 70.2970, GNorm = 0.3912, lr_0 = 6.7200e-04
Validation prc-auc = 0.352726
Epoch 240
Validation prc-auc = 0.362634
Epoch 241
Validation prc-auc = 0.355870
Epoch 242
Validation prc-auc = 0.348687
Epoch 243
Validation prc-auc = 0.347288
Epoch 244
Loss = 1.9669e-04, PNorm = 70.3309, GNorm = 0.1700, lr_0 = 6.6625e-04
Validation prc-auc = 0.345628
Epoch 245
Validation prc-auc = 0.356520
Epoch 246
Validation prc-auc = 0.384871
Epoch 247
Validation prc-auc = 0.406120
Epoch 248
Validation prc-auc = 0.432091
Epoch 249
Loss = 8.1460e-05, PNorm = 70.3592, GNorm = 0.4694, lr_0 = 6.6056e-04
Validation prc-auc = 0.433436
Epoch 250
Validation prc-auc = 0.436279
Epoch 251
Validation prc-auc = 0.436219
Epoch 252
Validation prc-auc = 0.436071
Epoch 253
Validation prc-auc = 0.456388
Epoch 254
Loss = 5.9212e-05, PNorm = 70.3890, GNorm = 0.2680, lr_0 = 6.5491e-04
Validation prc-auc = 0.456388
Epoch 255
Validation prc-auc = 0.459729
Epoch 256
Validation prc-auc = 0.448896
Epoch 257
Validation prc-auc = 0.425495
Epoch 258
Validation prc-auc = 0.393120
Epoch 259
Loss = 1.6843e-04, PNorm = 70.4087, GNorm = 0.0638, lr_0 = 6.4931e-04
Validation prc-auc = 0.298689
Epoch 260
Validation prc-auc = 0.288989
Epoch 261
Validation prc-auc = 0.282204
Epoch 262
Validation prc-auc = 0.283405
Epoch 263
Validation prc-auc = 0.280433
Epoch 264
Loss = 4.8956e-05, PNorm = 70.4296, GNorm = 0.1324, lr_0 = 6.4376e-04
Validation prc-auc = 0.277957
Epoch 265
Validation prc-auc = 0.281167
Epoch 266
Validation prc-auc = 0.304022
Epoch 267
Validation prc-auc = 0.312129
Epoch 268
Validation prc-auc = 0.308022
Epoch 269
Loss = 2.0921e-05, PNorm = 70.4728, GNorm = 0.0383, lr_0 = 6.3826e-04
Validation prc-auc = 0.305681
Epoch 270
Validation prc-auc = 0.310171
Epoch 271
Validation prc-auc = 0.385549
Epoch 272
Validation prc-auc = 0.382245
Epoch 273
Validation prc-auc = 0.386466
Epoch 274
Loss = 3.9504e-05, PNorm = 70.5294, GNorm = 0.0323, lr_0 = 6.3280e-04
Validation prc-auc = 0.407995
Epoch 275
Validation prc-auc = 0.407446
Epoch 276
Validation prc-auc = 0.411937
Epoch 277
Validation prc-auc = 0.411514
Epoch 278
Validation prc-auc = 0.414205
Epoch 279
Loss = 5.3776e-05, PNorm = 70.5592, GNorm = 0.3638, lr_0 = 6.2739e-04
Validation prc-auc = 0.425119
Epoch 280
Validation prc-auc = 0.420787
Epoch 281
Validation prc-auc = 0.413569
Epoch 282
Validation prc-auc = 0.413319
Epoch 283
Validation prc-auc = 0.413869
Epoch 284
Loss = 1.4229e-05, PNorm = 70.5820, GNorm = 0.1026, lr_0 = 6.2203e-04
Validation prc-auc = 0.414797
Epoch 285
Validation prc-auc = 0.418206
Epoch 286
Validation prc-auc = 0.421054
Epoch 287
Validation prc-auc = 0.420322
Epoch 288
Validation prc-auc = 0.419445
Epoch 289
Loss = 2.1808e-05, PNorm = 70.5941, GNorm = 0.0297, lr_0 = 6.1671e-04
Validation prc-auc = 0.421966
Epoch 290
Validation prc-auc = 0.403491
Epoch 291
Validation prc-auc = 0.402337
Epoch 292
Validation prc-auc = 0.404094
Epoch 293
Validation prc-auc = 0.404094
Epoch 294
Loss = 4.6875e-05, PNorm = 70.6013, GNorm = 0.3039, lr_0 = 6.1144e-04
Validation prc-auc = 0.402847
Epoch 295
Validation prc-auc = 0.397828
Epoch 296
Validation prc-auc = 0.397828
Epoch 297
Validation prc-auc = 0.398018
Epoch 298
Validation prc-auc = 0.397789
Epoch 299
Loss = 2.0487e-05, PNorm = 70.6074, GNorm = 0.0765, lr_0 = 6.0621e-04
Validation prc-auc = 0.397789
Model 0 best validation prc-auc = 0.497273 on epoch 60
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.536669
Ensemble test prc-auc = 0.536669
Fold 1
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_1',
 'save_smiles_splits': False,
 'seed': 1,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.381661
Best score so far is 0.381661 at epoch 0 
Epoch 1
Validation prc-auc = 0.382840
Best score so far is 0.382840 at epoch 1 
Epoch 2
Validation prc-auc = 0.403978
Best score so far is 0.403978 at epoch 2 
Epoch 3
Validation prc-auc = 0.361266
Epoch 4
Loss = 1.0538e-02, PNorm = 66.7733, GNorm = 5.0264, lr_0 = 6.5000e-04
Validation prc-auc = 0.356307
Epoch 5
Validation prc-auc = 0.358702
Epoch 6
Validation prc-auc = 0.437659
Best score so far is 0.437659 at epoch 6 
Epoch 7
Validation prc-auc = 0.436000
Epoch 8
Validation prc-auc = 0.413500
Epoch 9
Loss = 5.4429e-03, PNorm = 66.9312, GNorm = 2.4740, lr_0 = 9.9743e-04
Validation prc-auc = 0.411606
Epoch 10
Validation prc-auc = 0.285497
Epoch 11
Validation prc-auc = 0.390809
Epoch 12
Validation prc-auc = 0.440067
Best score so far is 0.440067 at epoch 12 
Epoch 13
Validation prc-auc = 0.433892
Epoch 14
Loss = 4.5524e-03, PNorm = 67.1203, GNorm = 2.0973, lr_0 = 9.8890e-04
Validation prc-auc = 0.313215
Epoch 15
Validation prc-auc = 0.259859
Epoch 16
Validation prc-auc = 0.288905
Epoch 17
Validation prc-auc = 0.344020
Epoch 18
Validation prc-auc = 0.404194
Epoch 19
Loss = 3.2647e-03, PNorm = 67.2835, GNorm = 2.0242, lr_0 = 9.8045e-04
Validation prc-auc = 0.450333
Best score so far is 0.450333 at epoch 19 
Epoch 20
Validation prc-auc = 0.486458
Best score so far is 0.486458 at epoch 20 
Epoch 21
Validation prc-auc = 0.445758
Epoch 22
Validation prc-auc = 0.424929
Epoch 23
Validation prc-auc = 0.420253
Epoch 24
Loss = 2.4803e-03, PNorm = 67.4204, GNorm = 1.6773, lr_0 = 9.7207e-04
Validation prc-auc = 0.325415
Epoch 25
Validation prc-auc = 0.334519
Epoch 26
Validation prc-auc = 0.332654
Epoch 27
Validation prc-auc = 0.367921
Epoch 28
Validation prc-auc = 0.386682
Epoch 29
Loss = 2.6962e-03, PNorm = 67.5423, GNorm = 1.6515, lr_0 = 9.6376e-04
Validation prc-auc = 0.387469
Epoch 30
Validation prc-auc = 0.395002
Epoch 31
Validation prc-auc = 0.411143
Epoch 32
Validation prc-auc = 0.394184
Epoch 33
Validation prc-auc = 0.369430
Epoch 34
Loss = 8.8934e-04, PNorm = 67.6555, GNorm = 0.8628, lr_0 = 9.5552e-04
Validation prc-auc = 0.309996
Epoch 35
Validation prc-auc = 0.216720
Epoch 36
Validation prc-auc = 0.246987
Epoch 37
Validation prc-auc = 0.372048
Epoch 38
Validation prc-auc = 0.368638
Epoch 39
Loss = 5.1220e-03, PNorm = 67.7601, GNorm = 3.2357, lr_0 = 9.4735e-04
Validation prc-auc = 0.356138
Epoch 40
Validation prc-auc = 0.331958
Epoch 41
Validation prc-auc = 0.364581
Epoch 42
Validation prc-auc = 0.367255
Epoch 43
Validation prc-auc = 0.363983
Epoch 44
Loss = 2.3691e-03, PNorm = 67.8422, GNorm = 0.8499, lr_0 = 9.3925e-04
Validation prc-auc = 0.378157
Epoch 45
Validation prc-auc = 0.406975
Epoch 46
Validation prc-auc = 0.391649
Epoch 47
Validation prc-auc = 0.361972
Epoch 48
Validation prc-auc = 0.352810
Epoch 49
Loss = 1.4481e-03, PNorm = 67.9308, GNorm = 0.9334, lr_0 = 9.3122e-04
Validation prc-auc = 0.347705
Epoch 50
Validation prc-auc = 0.327901
Epoch 51
Validation prc-auc = 0.351297
Epoch 52
Validation prc-auc = 0.347502
Epoch 53
Validation prc-auc = 0.349764
Epoch 54
Loss = 2.8628e-03, PNorm = 68.0062, GNorm = 2.1542, lr_0 = 9.2326e-04
Validation prc-auc = 0.355233
Epoch 55
Validation prc-auc = 0.369992
Epoch 56
Validation prc-auc = 0.365268
Epoch 57
Validation prc-auc = 0.416368
Epoch 58
Validation prc-auc = 0.424989
Epoch 59
Loss = 3.5828e-04, PNorm = 68.0717, GNorm = 0.2845, lr_0 = 9.1537e-04
Validation prc-auc = 0.443478
Epoch 60
Validation prc-auc = 0.441906
Epoch 61
Validation prc-auc = 0.477751
Epoch 62
Validation prc-auc = 0.455560
Epoch 63
Validation prc-auc = 0.413444
Epoch 64
Loss = 1.2337e-03, PNorm = 68.1452, GNorm = 0.6849, lr_0 = 9.0754e-04
Validation prc-auc = 0.344682
Epoch 65
Validation prc-auc = 0.329265
Epoch 66
Validation prc-auc = 0.369042
Epoch 67
Validation prc-auc = 0.398568
Epoch 68
Validation prc-auc = 0.431740
Epoch 69
Loss = 5.3730e-04, PNorm = 68.1998, GNorm = 0.9466, lr_0 = 8.9978e-04
Validation prc-auc = 0.457366
Epoch 70
Validation prc-auc = 0.441046
Epoch 71
Validation prc-auc = 0.449029
Epoch 72
Validation prc-auc = 0.406605
Epoch 73
Validation prc-auc = 0.387359
Epoch 74
Loss = 9.5229e-04, PNorm = 68.2563, GNorm = 0.2245, lr_0 = 8.9209e-04
Validation prc-auc = 0.265447
Epoch 75
Validation prc-auc = 0.250590
Epoch 76
Validation prc-auc = 0.246302
Epoch 77
Validation prc-auc = 0.348272
Epoch 78
Validation prc-auc = 0.455065
Epoch 79
Loss = 6.8961e-04, PNorm = 68.3335, GNorm = 1.4745, lr_0 = 8.8447e-04
Validation prc-auc = 0.460180
Epoch 80
Validation prc-auc = 0.456935
Epoch 81
Validation prc-auc = 0.437045
Epoch 82
Validation prc-auc = 0.405786
Epoch 83
Validation prc-auc = 0.386546
Epoch 84
Loss = 2.8480e-04, PNorm = 68.4309, GNorm = 0.4299, lr_0 = 8.7691e-04
Validation prc-auc = 0.347492
Epoch 85
Validation prc-auc = 0.336808
Epoch 86
Validation prc-auc = 0.259219
Epoch 87
Validation prc-auc = 0.269867
Epoch 88
Validation prc-auc = 0.339181
Epoch 89
Loss = 2.5396e-03, PNorm = 68.5042, GNorm = 2.8119, lr_0 = 8.6941e-04
Validation prc-auc = 0.465155
Epoch 90
Validation prc-auc = 0.494408
Best score so far is 0.494408 at epoch 90 
Epoch 91
Validation prc-auc = 0.517240
Best score so far is 0.517240 at epoch 91 
Epoch 92
Validation prc-auc = 0.512066
Epoch 93
Validation prc-auc = 0.474626
Epoch 94
Loss = 7.9048e-04, PNorm = 68.6136, GNorm = 1.9690, lr_0 = 8.6198e-04
Validation prc-auc = 0.397919
Epoch 95
Validation prc-auc = 0.298762
Epoch 96
Validation prc-auc = 0.262109
Epoch 97
Validation prc-auc = 0.246107
Epoch 98
Validation prc-auc = 0.250653
Epoch 99
Loss = 1.0324e-03, PNorm = 68.7505, GNorm = 0.2239, lr_0 = 8.5461e-04
Validation prc-auc = 0.311936
Epoch 100
Validation prc-auc = 0.364827
Epoch 101
Validation prc-auc = 0.437493
Epoch 102
Validation prc-auc = 0.432237
Epoch 103
Validation prc-auc = 0.425867
Epoch 104
Loss = 3.2713e-04, PNorm = 68.8782, GNorm = 0.1518, lr_0 = 8.4730e-04
Validation prc-auc = 0.429772
Epoch 105
Validation prc-auc = 0.368201
Epoch 106
Validation prc-auc = 0.368299
Epoch 107
Validation prc-auc = 0.393393
Epoch 108
Validation prc-auc = 0.413541
Epoch 109
Loss = 5.0192e-05, PNorm = 68.9567, GNorm = 0.1929, lr_0 = 8.4006e-04
Validation prc-auc = 0.465865
Epoch 110
Validation prc-auc = 0.461518
Epoch 111
Validation prc-auc = 0.470115
Epoch 112
Validation prc-auc = 0.435596
Epoch 113
Validation prc-auc = 0.409588
Epoch 114
Loss = 6.1053e-05, PNorm = 69.0240, GNorm = 0.0422, lr_0 = 8.3288e-04
Validation prc-auc = 0.408814
Epoch 115
Validation prc-auc = 0.386976
Epoch 116
Validation prc-auc = 0.381311
Epoch 117
Validation prc-auc = 0.377099
Epoch 118
Validation prc-auc = 0.393011
Epoch 119
Loss = 3.2002e-03, PNorm = 69.0771, GNorm = 3.0442, lr_0 = 8.2576e-04
Validation prc-auc = 0.404332
Epoch 120
Validation prc-auc = 0.403707
Epoch 121
Validation prc-auc = 0.409168
Epoch 122
Validation prc-auc = 0.365690
Epoch 123
Validation prc-auc = 0.341765
Epoch 124
Loss = 1.0034e-03, PNorm = 69.1333, GNorm = 0.5696, lr_0 = 8.1870e-04
Validation prc-auc = 0.340340
Epoch 125
Validation prc-auc = 0.349879
Epoch 126
Validation prc-auc = 0.385570
Epoch 127
Validation prc-auc = 0.421453
Epoch 128
Validation prc-auc = 0.449127
Epoch 129
Loss = 7.1243e-04, PNorm = 69.1982, GNorm = 1.9459, lr_0 = 8.1170e-04
Validation prc-auc = 0.469733
Epoch 130
Validation prc-auc = 0.486512
Epoch 131
Validation prc-auc = 0.463091
Epoch 132
Validation prc-auc = 0.437745
Epoch 133
Validation prc-auc = 0.348305
Epoch 134
Loss = 2.6026e-04, PNorm = 69.2752, GNorm = 0.2960, lr_0 = 8.0476e-04
Validation prc-auc = 0.340246
Epoch 135
Validation prc-auc = 0.328635
Epoch 136
Validation prc-auc = 0.316210
Epoch 137
Validation prc-auc = 0.316487
Epoch 138
Validation prc-auc = 0.311023
Epoch 139
Loss = 7.6891e-04, PNorm = 69.3633, GNorm = 0.9596, lr_0 = 7.9788e-04
Validation prc-auc = 0.395822
Epoch 140
Validation prc-auc = 0.394332
Epoch 141
Validation prc-auc = 0.410228
Epoch 142
Validation prc-auc = 0.401662
Epoch 143
Validation prc-auc = 0.390616
Epoch 144
Loss = 4.1789e-04, PNorm = 69.4579, GNorm = 1.2531, lr_0 = 7.9106e-04
Validation prc-auc = 0.403739
Epoch 145
Validation prc-auc = 0.417053
Epoch 146
Validation prc-auc = 0.421038
Epoch 147
Validation prc-auc = 0.428745
Epoch 148
Validation prc-auc = 0.393047
Epoch 149
Loss = 1.6373e-03, PNorm = 69.5546, GNorm = 2.7953, lr_0 = 7.8430e-04
Validation prc-auc = 0.300433
Epoch 150
Validation prc-auc = 0.320426
Epoch 151
Validation prc-auc = 0.309535
Epoch 152
Validation prc-auc = 0.300443
Epoch 153
Validation prc-auc = 0.302034
Epoch 154
Loss = 1.9819e-04, PNorm = 69.6618, GNorm = 0.3329, lr_0 = 7.7759e-04
Validation prc-auc = 0.313432
Epoch 155
Validation prc-auc = 0.310542
Epoch 156
Validation prc-auc = 0.322504
Epoch 157
Validation prc-auc = 0.312485
Epoch 158
Validation prc-auc = 0.318541
Epoch 159
Loss = 1.1445e-04, PNorm = 69.7477, GNorm = 0.0697, lr_0 = 7.7095e-04
Validation prc-auc = 0.310404
Epoch 160
Validation prc-auc = 0.344113
Epoch 161
Validation prc-auc = 0.378480
Epoch 162
Validation prc-auc = 0.379512
Epoch 163
Validation prc-auc = 0.426713
Epoch 164
Loss = 7.4757e-04, PNorm = 69.8085, GNorm = 0.4487, lr_0 = 7.6436e-04
Validation prc-auc = 0.409663
Epoch 165
Validation prc-auc = 0.405816
Epoch 166
Validation prc-auc = 0.392346
Epoch 167
Validation prc-auc = 0.383243
Epoch 168
Validation prc-auc = 0.364433
Epoch 169
Loss = 4.4206e-04, PNorm = 69.8681, GNorm = 0.7862, lr_0 = 7.5782e-04
Validation prc-auc = 0.353313
Epoch 170
Validation prc-auc = 0.328027
Epoch 171
Validation prc-auc = 0.291785
Epoch 172
Validation prc-auc = 0.305070
Epoch 173
Validation prc-auc = 0.308689
Epoch 174
Loss = 2.7141e-04, PNorm = 69.9342, GNorm = 0.6505, lr_0 = 7.5134e-04
Validation prc-auc = 0.315612
Epoch 175
Validation prc-auc = 0.311958
Epoch 176
Validation prc-auc = 0.268517
Epoch 177
Validation prc-auc = 0.266594
Epoch 178
Validation prc-auc = 0.275893
Epoch 179
Loss = 8.2382e-04, PNorm = 70.0228, GNorm = 1.0998, lr_0 = 7.4492e-04
Validation prc-auc = 0.347602
Epoch 180
Validation prc-auc = 0.382429
Epoch 181
Validation prc-auc = 0.398460
Epoch 182
Validation prc-auc = 0.401232
Epoch 183
Validation prc-auc = 0.406376
Epoch 184
Loss = 1.8840e-04, PNorm = 70.1019, GNorm = 0.3650, lr_0 = 7.3855e-04
Validation prc-auc = 0.414776
Epoch 185
Validation prc-auc = 0.407549
Epoch 186
Validation prc-auc = 0.419320
Epoch 187
Validation prc-auc = 0.415218
Epoch 188
Validation prc-auc = 0.412014
Epoch 189
Loss = 2.6037e-04, PNorm = 70.1492, GNorm = 0.7774, lr_0 = 7.3224e-04
Validation prc-auc = 0.397109
Epoch 190
Validation prc-auc = 0.395148
Epoch 191
Validation prc-auc = 0.387415
Epoch 192
Validation prc-auc = 0.385703
Epoch 193
Validation prc-auc = 0.389163
Epoch 194
Loss = 1.1428e-03, PNorm = 70.2101, GNorm = 0.7777, lr_0 = 7.2598e-04
Validation prc-auc = 0.395895
Epoch 195
Validation prc-auc = 0.409539
Epoch 196
Validation prc-auc = 0.420321
Epoch 197
Validation prc-auc = 0.403119
Epoch 198
Validation prc-auc = 0.395691
Epoch 199
Loss = 3.0619e-04, PNorm = 70.2740, GNorm = 0.2699, lr_0 = 7.1977e-04
Validation prc-auc = 0.370771
Epoch 200
Validation prc-auc = 0.365314
Epoch 201
Validation prc-auc = 0.358491
Epoch 202
Validation prc-auc = 0.370941
Epoch 203
Validation prc-auc = 0.385590
Epoch 204
Loss = 7.4912e-04, PNorm = 70.3325, GNorm = 1.5536, lr_0 = 7.1362e-04
Validation prc-auc = 0.399813
Epoch 205
Validation prc-auc = 0.396959
Epoch 206
Validation prc-auc = 0.390238
Epoch 207
Validation prc-auc = 0.385147
Epoch 208
Validation prc-auc = 0.384076
Epoch 209
Loss = 2.4535e-03, PNorm = 70.4002, GNorm = 0.3719, lr_0 = 7.0752e-04
Validation prc-auc = 0.382581
Epoch 210
Validation prc-auc = 0.387686
Epoch 211
Validation prc-auc = 0.387674
Epoch 212
Validation prc-auc = 0.384083
Epoch 213
Validation prc-auc = 0.381585
Epoch 214
Loss = 4.5732e-04, PNorm = 70.4391, GNorm = 0.7514, lr_0 = 7.0147e-04
Validation prc-auc = 0.396983
Epoch 215
Validation prc-auc = 0.417071
Epoch 216
Validation prc-auc = 0.422209
Epoch 217
Validation prc-auc = 0.412894
Epoch 218
Validation prc-auc = 0.454575
Epoch 219
Loss = 1.9985e-04, PNorm = 70.4734, GNorm = 0.9013, lr_0 = 6.9548e-04
Validation prc-auc = 0.435711
Epoch 220
Validation prc-auc = 0.456110
Epoch 221
Validation prc-auc = 0.456548
Epoch 222
Validation prc-auc = 0.466983
Epoch 223
Validation prc-auc = 0.474639
Epoch 224
Loss = 8.7625e-05, PNorm = 70.5019, GNorm = 0.0860, lr_0 = 6.8953e-04
Validation prc-auc = 0.475597
Epoch 225
Validation prc-auc = 0.470743
Epoch 226
Validation prc-auc = 0.459655
Epoch 227
Validation prc-auc = 0.459975
Epoch 228
Validation prc-auc = 0.456182
Epoch 229
Loss = 4.6343e-05, PNorm = 70.5191, GNorm = 0.1289, lr_0 = 6.8364e-04
Validation prc-auc = 0.456397
Epoch 230
Validation prc-auc = 0.458929
Epoch 231
Validation prc-auc = 0.446336
Epoch 232
Validation prc-auc = 0.439523
Epoch 233
Validation prc-auc = 0.441414
Epoch 234
Loss = 1.3509e-04, PNorm = 70.5342, GNorm = 0.4006, lr_0 = 6.7779e-04
Validation prc-auc = 0.440646
Epoch 235
Validation prc-auc = 0.440115
Epoch 236
Validation prc-auc = 0.460640
Epoch 237
Validation prc-auc = 0.467210
Epoch 238
Validation prc-auc = 0.472290
Epoch 239
Loss = 1.6458e-04, PNorm = 70.5590, GNorm = 0.4250, lr_0 = 6.7200e-04
Validation prc-auc = 0.471563
Epoch 240
Validation prc-auc = 0.467626
Epoch 241
Validation prc-auc = 0.465933
Epoch 242
Validation prc-auc = 0.465917
Epoch 243
Validation prc-auc = 0.458030
Epoch 244
Loss = 2.2254e-05, PNorm = 70.5791, GNorm = 0.0473, lr_0 = 6.6625e-04
Validation prc-auc = 0.455959
Epoch 245
Validation prc-auc = 0.454568
Epoch 246
Validation prc-auc = 0.447794
Epoch 247
Validation prc-auc = 0.441910
Epoch 248
Validation prc-auc = 0.442288
Epoch 249
Loss = 2.0021e-05, PNorm = 70.5942, GNorm = 0.0210, lr_0 = 6.6056e-04
Validation prc-auc = 0.436764
Epoch 250
Validation prc-auc = 0.437873
Epoch 251
Validation prc-auc = 0.436661
Epoch 252
Validation prc-auc = 0.430597
Epoch 253
Validation prc-auc = 0.428002
Epoch 254
Loss = 5.3677e-05, PNorm = 70.6160, GNorm = 0.0867, lr_0 = 6.5491e-04
Validation prc-auc = 0.403241
Epoch 255
Validation prc-auc = 0.401920
Epoch 256
Validation prc-auc = 0.401492
Epoch 257
Validation prc-auc = 0.417375
Epoch 258
Validation prc-auc = 0.414130
Epoch 259
Loss = 8.8539e-05, PNorm = 70.6497, GNorm = 0.1885, lr_0 = 6.4931e-04
Validation prc-auc = 0.413669
Epoch 260
Validation prc-auc = 0.417135
Epoch 261
Validation prc-auc = 0.419307
Epoch 262
Validation prc-auc = 0.421270
Epoch 263
Validation prc-auc = 0.425023
Epoch 264
Loss = 7.7780e-05, PNorm = 70.6689, GNorm = 0.0012, lr_0 = 6.4376e-04
Validation prc-auc = 0.426470
Epoch 265
Validation prc-auc = 0.428561
Epoch 266
Validation prc-auc = 0.426043
Epoch 267
Validation prc-auc = 0.425604
Epoch 268
Validation prc-auc = 0.427227
Epoch 269
Loss = 1.3947e-05, PNorm = 70.6918, GNorm = 0.0082, lr_0 = 6.3826e-04
Validation prc-auc = 0.424262
Epoch 270
Validation prc-auc = 0.422200
Epoch 271
Validation prc-auc = 0.416551
Epoch 272
Validation prc-auc = 0.418270
Epoch 273
Validation prc-auc = 0.424533
Epoch 274
Loss = 3.7224e-05, PNorm = 70.7323, GNorm = 0.2605, lr_0 = 6.3280e-04
Validation prc-auc = 0.417059
Epoch 275
Validation prc-auc = 0.410892
Epoch 276
Validation prc-auc = 0.392818
Epoch 277
Validation prc-auc = 0.397754
Epoch 278
Validation prc-auc = 0.421090
Epoch 279
Loss = 1.4837e-05, PNorm = 70.7520, GNorm = 0.0046, lr_0 = 6.2739e-04
Validation prc-auc = 0.427568
Epoch 280
Validation prc-auc = 0.432275
Epoch 281
Validation prc-auc = 0.430804
Epoch 282
Validation prc-auc = 0.435900
Epoch 283
Validation prc-auc = 0.432085
Epoch 284
Loss = 2.7964e-05, PNorm = 70.7651, GNorm = 0.0220, lr_0 = 6.2203e-04
Validation prc-auc = 0.429819
Epoch 285
Validation prc-auc = 0.427299
Epoch 286
Validation prc-auc = 0.422046
Epoch 287
Validation prc-auc = 0.392827
Epoch 288
Validation prc-auc = 0.394020
Epoch 289
Loss = 2.6283e-05, PNorm = 70.7885, GNorm = 0.0871, lr_0 = 6.1671e-04
Validation prc-auc = 0.391600
Epoch 290
Validation prc-auc = 0.387989
Epoch 291
Validation prc-auc = 0.369645
Epoch 292
Validation prc-auc = 0.358371
Epoch 293
Validation prc-auc = 0.346004
Epoch 294
Loss = 2.3763e-05, PNorm = 70.8071, GNorm = 0.0491, lr_0 = 6.1144e-04
Validation prc-auc = 0.342442
Epoch 295
Validation prc-auc = 0.347022
Epoch 296
Validation prc-auc = 0.352374
Epoch 297
Validation prc-auc = 0.355011
Epoch 298
Validation prc-auc = 0.359403
Epoch 299
Loss = 3.1710e-04, PNorm = 70.8165, GNorm = 2.2116, lr_0 = 6.0621e-04
Validation prc-auc = 0.360467
Model 0 best validation prc-auc = 0.517240 on epoch 91
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.360351
Ensemble test prc-auc = 0.360351
Fold 2
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_2',
 'save_smiles_splits': False,
 'seed': 2,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.471943
Best score so far is 0.471943 at epoch 0 
Epoch 1
Validation prc-auc = 0.393548
Epoch 2
Validation prc-auc = 0.391342
Epoch 3
Validation prc-auc = 0.360199
Epoch 4
Loss = 9.9587e-03, PNorm = 66.7754, GNorm = 3.3174, lr_0 = 6.5000e-04
Validation prc-auc = 0.299689
Epoch 5
Validation prc-auc = 0.237357
Epoch 6
Validation prc-auc = 0.268841
Epoch 7
Validation prc-auc = 0.283171
Epoch 8
Validation prc-auc = 0.381313
Epoch 9
Loss = 7.3704e-03, PNorm = 66.9237, GNorm = 2.5960, lr_0 = 9.9743e-04
Validation prc-auc = 0.333337
Epoch 10
Validation prc-auc = 0.286034
Epoch 11
Validation prc-auc = 0.261274
Epoch 12
Validation prc-auc = 0.240850
Epoch 13
Validation prc-auc = 0.342334
Epoch 14
Loss = 5.0000e-03, PNorm = 67.1146, GNorm = 3.9097, lr_0 = 9.8890e-04
Validation prc-auc = 0.353627
Epoch 15
Validation prc-auc = 0.306575
Epoch 16
Validation prc-auc = 0.349666
Epoch 17
Validation prc-auc = 0.350770
Epoch 18
Validation prc-auc = 0.331980
Epoch 19
Loss = 2.7895e-03, PNorm = 67.2718, GNorm = 1.4058, lr_0 = 9.8045e-04
Validation prc-auc = 0.307012
Epoch 20
Validation prc-auc = 0.333122
Epoch 21
Validation prc-auc = 0.389625
Epoch 22
Validation prc-auc = 0.373516
Epoch 23
Validation prc-auc = 0.327272
Epoch 24
Loss = 3.7095e-03, PNorm = 67.3988, GNorm = 1.0785, lr_0 = 9.7207e-04
Validation prc-auc = 0.347605
Epoch 25
Validation prc-auc = 0.378717
Epoch 26
Validation prc-auc = 0.393685
Epoch 27
Validation prc-auc = 0.401609
Epoch 28
Validation prc-auc = 0.395340
Epoch 29
Loss = 5.9130e-03, PNorm = 67.5208, GNorm = 2.7134, lr_0 = 9.6376e-04
Validation prc-auc = 0.400229
Epoch 30
Validation prc-auc = 0.383140
Epoch 31
Validation prc-auc = 0.344845
Epoch 32
Validation prc-auc = 0.258023
Epoch 33
Validation prc-auc = 0.288466
Epoch 34
Loss = 3.0096e-03, PNorm = 67.6253, GNorm = 1.2291, lr_0 = 9.5552e-04
Validation prc-auc = 0.307773
Epoch 35
Validation prc-auc = 0.314447
Epoch 36
Validation prc-auc = 0.390651
Epoch 37
Validation prc-auc = 0.408055
Epoch 38
Validation prc-auc = 0.420105
Epoch 39
Loss = 1.9407e-03, PNorm = 67.7227, GNorm = 1.8359, lr_0 = 9.4735e-04
Validation prc-auc = 0.418325
Epoch 40
Validation prc-auc = 0.382313
Epoch 41
Validation prc-auc = 0.310916
Epoch 42
Validation prc-auc = 0.307800
Epoch 43
Validation prc-auc = 0.357356
Epoch 44
Loss = 2.3226e-03, PNorm = 67.8163, GNorm = 0.4214, lr_0 = 9.3925e-04
Validation prc-auc = 0.395730
Epoch 45
Validation prc-auc = 0.434289
Epoch 46
Validation prc-auc = 0.457170
Epoch 47
Validation prc-auc = 0.498558
Best score so far is 0.498558 at epoch 47 
Epoch 48
Validation prc-auc = 0.461407
Epoch 49
Loss = 1.3209e-03, PNorm = 67.9147, GNorm = 1.5611, lr_0 = 9.3122e-04
Validation prc-auc = 0.417824
Epoch 50
Validation prc-auc = 0.296060
Epoch 51
Validation prc-auc = 0.293536
Epoch 52
Validation prc-auc = 0.254172
Epoch 53
Validation prc-auc = 0.242941
Epoch 54
Loss = 1.3081e-03, PNorm = 68.0144, GNorm = 0.2752, lr_0 = 9.2326e-04
Validation prc-auc = 0.338116
Epoch 55
Validation prc-auc = 0.391172
Epoch 56
Validation prc-auc = 0.409621
Epoch 57
Validation prc-auc = 0.409533
Epoch 58
Validation prc-auc = 0.394960
Epoch 59
Loss = 2.5674e-03, PNorm = 68.1188, GNorm = 3.1765, lr_0 = 9.1537e-04
Validation prc-auc = 0.375946
Epoch 60
Validation prc-auc = 0.356931
Epoch 61
Validation prc-auc = 0.313900
Epoch 62
Validation prc-auc = 0.307812
Epoch 63
Validation prc-auc = 0.328093
Epoch 64
Loss = 1.2439e-03, PNorm = 68.2168, GNorm = 2.1350, lr_0 = 9.0754e-04
Validation prc-auc = 0.370065
Epoch 65
Validation prc-auc = 0.404421
Epoch 66
Validation prc-auc = 0.403618
Epoch 67
Validation prc-auc = 0.392736
Epoch 68
Validation prc-auc = 0.320226
Epoch 69
Loss = 4.4807e-04, PNorm = 68.3127, GNorm = 0.4624, lr_0 = 8.9978e-04
Validation prc-auc = 0.297761
Epoch 70
Validation prc-auc = 0.273800
Epoch 71
Validation prc-auc = 0.333424
Epoch 72
Validation prc-auc = 0.330277
Epoch 73
Validation prc-auc = 0.333578
Epoch 74
Loss = 1.3473e-03, PNorm = 68.4298, GNorm = 0.9428, lr_0 = 8.9209e-04
Validation prc-auc = 0.354926
Epoch 75
Validation prc-auc = 0.379736
Epoch 76
Validation prc-auc = 0.392864
Epoch 77
Validation prc-auc = 0.377428
Epoch 78
Validation prc-auc = 0.355707
Epoch 79
Loss = 4.3319e-03, PNorm = 68.5009, GNorm = 3.2088, lr_0 = 8.8447e-04
Validation prc-auc = 0.372096
Epoch 80
Validation prc-auc = 0.383086
Epoch 81
Validation prc-auc = 0.409798
Epoch 82
Validation prc-auc = 0.417354
Epoch 83
Validation prc-auc = 0.384351
Epoch 84
Loss = 6.1039e-04, PNorm = 68.6073, GNorm = 0.9024, lr_0 = 8.7691e-04
Validation prc-auc = 0.368406
Epoch 85
Validation prc-auc = 0.379490
Epoch 86
Validation prc-auc = 0.382043
Epoch 87
Validation prc-auc = 0.391826
Epoch 88
Validation prc-auc = 0.398321
Epoch 89
Loss = 6.9831e-04, PNorm = 68.6963, GNorm = 0.6072, lr_0 = 8.6941e-04
Validation prc-auc = 0.404147
Epoch 90
Validation prc-auc = 0.403417
Epoch 91
Validation prc-auc = 0.412783
Epoch 92
Validation prc-auc = 0.411320
Epoch 93
Validation prc-auc = 0.403667
Epoch 94
Loss = 2.7314e-04, PNorm = 68.7543, GNorm = 0.4292, lr_0 = 8.6198e-04
Validation prc-auc = 0.407245
Epoch 95
Validation prc-auc = 0.399923
Epoch 96
Validation prc-auc = 0.355577
Epoch 97
Validation prc-auc = 0.353835
Epoch 98
Validation prc-auc = 0.341975
Epoch 99
Loss = 1.7872e-04, PNorm = 68.7997, GNorm = 0.3471, lr_0 = 8.5461e-04
Validation prc-auc = 0.341813
Epoch 100
Validation prc-auc = 0.339353
Epoch 101
Validation prc-auc = 0.344196
Epoch 102
Validation prc-auc = 0.352344
Epoch 103
Validation prc-auc = 0.364443
Epoch 104
Loss = 3.0677e-04, PNorm = 68.8437, GNorm = 0.5872, lr_0 = 8.4730e-04
Validation prc-auc = 0.369852
Epoch 105
Validation prc-auc = 0.367615
Epoch 106
Validation prc-auc = 0.376376
Epoch 107
Validation prc-auc = 0.376211
Epoch 108
Validation prc-auc = 0.382221
Epoch 109
Loss = 1.7245e-04, PNorm = 68.8948, GNorm = 0.3674, lr_0 = 8.4006e-04
Validation prc-auc = 0.406162
Epoch 110
Validation prc-auc = 0.418264
Epoch 111
Validation prc-auc = 0.412933
Epoch 112
Validation prc-auc = 0.410697
Epoch 113
Validation prc-auc = 0.393428
Epoch 114
Loss = 6.9069e-04, PNorm = 68.9733, GNorm = 0.4521, lr_0 = 8.3288e-04
Validation prc-auc = 0.394033
Epoch 115
Validation prc-auc = 0.409912
Epoch 116
Validation prc-auc = 0.431669
Epoch 117
Validation prc-auc = 0.437208
Epoch 118
Validation prc-auc = 0.438645
Epoch 119
Loss = 9.2094e-04, PNorm = 69.0503, GNorm = 0.5729, lr_0 = 8.2576e-04
Validation prc-auc = 0.435901
Epoch 120
Validation prc-auc = 0.403856
Epoch 121
Validation prc-auc = 0.389671
Epoch 122
Validation prc-auc = 0.369904
Epoch 123
Validation prc-auc = 0.370131
Epoch 124
Loss = 3.8853e-04, PNorm = 69.1245, GNorm = 0.2018, lr_0 = 8.1870e-04
Validation prc-auc = 0.371547
Epoch 125
Validation prc-auc = 0.368105
Epoch 126
Validation prc-auc = 0.365764
Epoch 127
Validation prc-auc = 0.372763
Epoch 128
Validation prc-auc = 0.363929
Epoch 129
Loss = 4.8753e-04, PNorm = 69.1730, GNorm = 0.1722, lr_0 = 8.1170e-04
Validation prc-auc = 0.366901
Epoch 130
Validation prc-auc = 0.357303
Epoch 131
Validation prc-auc = 0.364622
Epoch 132
Validation prc-auc = 0.366575
Epoch 133
Validation prc-auc = 0.365335
Epoch 134
Loss = 6.2086e-04, PNorm = 69.2472, GNorm = 1.6514, lr_0 = 8.0476e-04
Validation prc-auc = 0.357452
Epoch 135
Validation prc-auc = 0.364446
Epoch 136
Validation prc-auc = 0.373845
Epoch 137
Validation prc-auc = 0.399979
Epoch 138
Validation prc-auc = 0.401258
Epoch 139
Loss = 1.3834e-03, PNorm = 69.3347, GNorm = 1.8958, lr_0 = 7.9788e-04
Validation prc-auc = 0.409541
Epoch 140
Validation prc-auc = 0.407357
Epoch 141
Validation prc-auc = 0.377699
Epoch 142
Validation prc-auc = 0.366167
Epoch 143
Validation prc-auc = 0.347750
Epoch 144
Loss = 9.3914e-04, PNorm = 69.4353, GNorm = 0.5365, lr_0 = 7.9106e-04
Validation prc-auc = 0.333141
Epoch 145
Validation prc-auc = 0.347042
Epoch 146
Validation prc-auc = 0.397763
Epoch 147
Validation prc-auc = 0.420946
Epoch 148
Validation prc-auc = 0.440999
Epoch 149
Loss = 3.7791e-04, PNorm = 69.5027, GNorm = 1.3157, lr_0 = 7.8430e-04
Validation prc-auc = 0.464570
Epoch 150
Validation prc-auc = 0.480928
Epoch 151
Validation prc-auc = 0.489278
Epoch 152
Validation prc-auc = 0.478577
Epoch 153
Validation prc-auc = 0.484583
Epoch 154
Loss = 3.3831e-04, PNorm = 69.5792, GNorm = 0.1332, lr_0 = 7.7759e-04
Validation prc-auc = 0.479821
Epoch 155
Validation prc-auc = 0.453870
Epoch 156
Validation prc-auc = 0.423866
Epoch 157
Validation prc-auc = 0.407912
Epoch 158
Validation prc-auc = 0.381897
Epoch 159
Loss = 2.4238e-04, PNorm = 69.6310, GNorm = 0.0973, lr_0 = 7.7095e-04
Validation prc-auc = 0.406331
Epoch 160
Validation prc-auc = 0.389416
Epoch 161
Validation prc-auc = 0.404299
Epoch 162
Validation prc-auc = 0.409551
Epoch 163
Validation prc-auc = 0.407655
Epoch 164
Loss = 5.9160e-05, PNorm = 69.6764, GNorm = 0.0894, lr_0 = 7.6436e-04
Validation prc-auc = 0.390767
Epoch 165
Validation prc-auc = 0.374695
Epoch 166
Validation prc-auc = 0.391614
Epoch 167
Validation prc-auc = 0.422228
Epoch 168
Validation prc-auc = 0.434094
Epoch 169
Loss = 7.0005e-04, PNorm = 69.7267, GNorm = 0.5646, lr_0 = 7.5782e-04
Validation prc-auc = 0.445478
Epoch 170
Validation prc-auc = 0.475122
Epoch 171
Validation prc-auc = 0.475939
Epoch 172
Validation prc-auc = 0.480093
Epoch 173
Validation prc-auc = 0.480186
Epoch 174
Loss = 8.4574e-05, PNorm = 69.7865, GNorm = 0.0825, lr_0 = 7.5134e-04
Validation prc-auc = 0.475440
Epoch 175
Validation prc-auc = 0.465523
Epoch 176
Validation prc-auc = 0.462516
Epoch 177
Validation prc-auc = 0.456944
Epoch 178
Validation prc-auc = 0.451088
Epoch 179
Loss = 2.2441e-04, PNorm = 69.8235, GNorm = 1.2677, lr_0 = 7.4492e-04
Validation prc-auc = 0.454900
Epoch 180
Validation prc-auc = 0.461516
Epoch 181
Validation prc-auc = 0.472304
Epoch 182
Validation prc-auc = 0.471372
Epoch 183
Validation prc-auc = 0.468830
Epoch 184
Loss = 1.5802e-04, PNorm = 69.8599, GNorm = 0.0137, lr_0 = 7.3855e-04
Validation prc-auc = 0.452267
Epoch 185
Validation prc-auc = 0.454855
Epoch 186
Validation prc-auc = 0.456503
Epoch 187
Validation prc-auc = 0.463273
Epoch 188
Validation prc-auc = 0.462516
Epoch 189
Loss = 3.1130e-05, PNorm = 69.8874, GNorm = 0.0921, lr_0 = 7.3224e-04
Validation prc-auc = 0.469418
Epoch 190
Validation prc-auc = 0.473275
Epoch 191
Validation prc-auc = 0.467716
Epoch 192
Validation prc-auc = 0.474878
Epoch 193
Validation prc-auc = 0.469770
Epoch 194
Loss = 8.0217e-04, PNorm = 69.9046, GNorm = 1.5939, lr_0 = 7.2598e-04
Validation prc-auc = 0.460139
Epoch 195
Validation prc-auc = 0.453037
Epoch 196
Validation prc-auc = 0.437258
Epoch 197
Validation prc-auc = 0.432967
Epoch 198
Validation prc-auc = 0.431002
Epoch 199
Loss = 3.0672e-05, PNorm = 69.9330, GNorm = 0.0407, lr_0 = 7.1977e-04
Validation prc-auc = 0.426768
Epoch 200
Validation prc-auc = 0.433540
Epoch 201
Validation prc-auc = 0.441092
Epoch 202
Validation prc-auc = 0.453912
Epoch 203
Validation prc-auc = 0.461473
Epoch 204
Loss = 6.4913e-04, PNorm = 69.9564, GNorm = 2.9851, lr_0 = 7.1362e-04
Validation prc-auc = 0.463909
Epoch 205
Validation prc-auc = 0.456452
Epoch 206
Validation prc-auc = 0.437444
Epoch 207
Validation prc-auc = 0.421660
Epoch 208
Validation prc-auc = 0.412419
Epoch 209
Loss = 2.7065e-05, PNorm = 70.0035, GNorm = 0.0530, lr_0 = 7.0752e-04
Validation prc-auc = 0.403060
Epoch 210
Validation prc-auc = 0.401195
Epoch 211
Validation prc-auc = 0.403839
Epoch 212
Validation prc-auc = 0.403428
Epoch 213
Validation prc-auc = 0.399167
Epoch 214
Loss = 3.0238e-05, PNorm = 70.0338, GNorm = 0.0358, lr_0 = 7.0147e-04
Validation prc-auc = 0.406505
Epoch 215
Validation prc-auc = 0.397944
Epoch 216
Validation prc-auc = 0.399790
Epoch 217
Validation prc-auc = 0.399206
Epoch 218
Validation prc-auc = 0.403120
Epoch 219
Loss = 2.1742e-05, PNorm = 70.0660, GNorm = 0.0144, lr_0 = 6.9548e-04
Validation prc-auc = 0.399701
Epoch 220
Validation prc-auc = 0.398400
Epoch 221
Validation prc-auc = 0.395476
Epoch 222
Validation prc-auc = 0.388413
Epoch 223
Validation prc-auc = 0.387972
Epoch 224
Loss = 1.3841e-04, PNorm = 70.0982, GNorm = 0.0756, lr_0 = 6.8953e-04
Validation prc-auc = 0.406032
Epoch 225
Validation prc-auc = 0.410359
Epoch 226
Validation prc-auc = 0.420804
Epoch 227
Validation prc-auc = 0.437181
Epoch 228
Validation prc-auc = 0.444677
Epoch 229
Loss = 1.4347e-04, PNorm = 70.1247, GNorm = 0.7702, lr_0 = 6.8364e-04
Validation prc-auc = 0.450866
Epoch 230
Validation prc-auc = 0.467909
Epoch 231
Validation prc-auc = 0.475474
Epoch 232
Validation prc-auc = 0.475695
Epoch 233
Validation prc-auc = 0.489246
Epoch 234
Loss = 3.1255e-05, PNorm = 70.1417, GNorm = 0.1043, lr_0 = 6.7779e-04
Validation prc-auc = 0.489117
Epoch 235
Validation prc-auc = 0.490421
Epoch 236
Validation prc-auc = 0.505777
Best score so far is 0.505777 at epoch 236 
Epoch 237
Validation prc-auc = 0.496840
Epoch 238
Validation prc-auc = 0.496947
Epoch 239
Loss = 1.9924e-05, PNorm = 70.1921, GNorm = 0.0903, lr_0 = 6.7200e-04
Validation prc-auc = 0.495515
Epoch 240
Validation prc-auc = 0.481129
Epoch 241
Validation prc-auc = 0.474565
Epoch 242
Validation prc-auc = 0.492372
Epoch 243
Validation prc-auc = 0.487578
Epoch 244
Loss = 3.2077e-05, PNorm = 70.2364, GNorm = 0.1368, lr_0 = 6.6625e-04
Validation prc-auc = 0.487819
Epoch 245
Validation prc-auc = 0.476554
Epoch 246
Validation prc-auc = 0.467084
Epoch 247
Validation prc-auc = 0.458333
Epoch 248
Validation prc-auc = 0.429083
Epoch 249
Loss = 1.5182e-04, PNorm = 70.2992, GNorm = 0.9123, lr_0 = 6.6056e-04
Validation prc-auc = 0.434105
Epoch 250
Validation prc-auc = 0.460143
Epoch 251
Validation prc-auc = 0.439667
Epoch 252
Validation prc-auc = 0.428607
Epoch 253
Validation prc-auc = 0.403960
Epoch 254
Loss = 2.6237e-04, PNorm = 70.4242, GNorm = 0.4871, lr_0 = 6.5491e-04
Validation prc-auc = 0.392451
Epoch 255
Validation prc-auc = 0.396024
Epoch 256
Validation prc-auc = 0.403814
Epoch 257
Validation prc-auc = 0.444674
Epoch 258
Validation prc-auc = 0.459885
Epoch 259
Loss = 1.2263e-05, PNorm = 70.5059, GNorm = 0.0225, lr_0 = 6.4931e-04
Validation prc-auc = 0.459341
Epoch 260
Validation prc-auc = 0.463506
Epoch 261
Validation prc-auc = 0.470244
Epoch 262
Validation prc-auc = 0.460126
Epoch 263
Validation prc-auc = 0.458842
Epoch 264
Loss = 7.6082e-06, PNorm = 70.5745, GNorm = 0.0153, lr_0 = 6.4376e-04
Validation prc-auc = 0.459723
Epoch 265
Validation prc-auc = 0.430174
Epoch 266
Validation prc-auc = 0.442064
Epoch 267
Validation prc-auc = 0.438968
Epoch 268
Validation prc-auc = 0.451555
Epoch 269
Loss = 6.9551e-05, PNorm = 70.6171, GNorm = 0.4147, lr_0 = 6.3826e-04
Validation prc-auc = 0.455214
Epoch 270
Validation prc-auc = 0.424590
Epoch 271
Validation prc-auc = 0.429971
Epoch 272
Validation prc-auc = 0.431353
Epoch 273
Validation prc-auc = 0.437352
Epoch 274
Loss = 1.3360e-03, PNorm = 70.6609, GNorm = 0.4622, lr_0 = 6.3280e-04
Validation prc-auc = 0.437466
Epoch 275
Validation prc-auc = 0.472734
Epoch 276
Validation prc-auc = 0.478781
Epoch 277
Validation prc-auc = 0.479818
Epoch 278
Validation prc-auc = 0.484457
Epoch 279
Loss = 1.2830e-04, PNorm = 70.7174, GNorm = 0.7036, lr_0 = 6.2739e-04
Validation prc-auc = 0.458075
Epoch 280
Validation prc-auc = 0.457554
Epoch 281
Validation prc-auc = 0.456493
Epoch 282
Validation prc-auc = 0.456274
Epoch 283
Validation prc-auc = 0.465269
Epoch 284
Loss = 3.7872e-05, PNorm = 70.7680, GNorm = 0.0588, lr_0 = 6.2203e-04
Validation prc-auc = 0.419820
Epoch 285
Validation prc-auc = 0.374992
Epoch 286
Validation prc-auc = 0.325132
Epoch 287
Validation prc-auc = 0.326438
Epoch 288
Validation prc-auc = 0.331873
Epoch 289
Loss = 4.9485e-05, PNorm = 70.8113, GNorm = 0.0705, lr_0 = 6.1671e-04
Validation prc-auc = 0.342544
Epoch 290
Validation prc-auc = 0.346909
Epoch 291
Validation prc-auc = 0.355642
Epoch 292
Validation prc-auc = 0.357438
Epoch 293
Validation prc-auc = 0.417476
Epoch 294
Loss = 3.8239e-04, PNorm = 70.8583, GNorm = 1.0309, lr_0 = 6.1144e-04
Validation prc-auc = 0.447096
Epoch 295
Validation prc-auc = 0.469137
Epoch 296
Validation prc-auc = 0.488789
Epoch 297
Validation prc-auc = 0.484266
Epoch 298
Validation prc-auc = 0.482738
Epoch 299
Loss = 3.8921e-05, PNorm = 70.8957, GNorm = 0.1311, lr_0 = 6.0621e-04
Validation prc-auc = 0.458124
Model 0 best validation prc-auc = 0.505777 on epoch 236
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.458042
Ensemble test prc-auc = 0.458042
Fold 3
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_3',
 'save_smiles_splits': False,
 'seed': 3,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.481082
Best score so far is 0.481082 at epoch 0 
Epoch 1
Validation prc-auc = 0.476636
Epoch 2
Validation prc-auc = 0.460712
Epoch 3
Validation prc-auc = 0.452154
Epoch 4
Loss = 8.8931e-03, PNorm = 66.7746, GNorm = 3.3674, lr_0 = 6.5000e-04
Validation prc-auc = 0.445964
Epoch 5
Validation prc-auc = 0.426299
Epoch 6
Validation prc-auc = 0.323470
Epoch 7
Validation prc-auc = 0.391408
Epoch 8
Validation prc-auc = 0.411995
Epoch 9
Loss = 5.6443e-03, PNorm = 66.9165, GNorm = 1.6404, lr_0 = 9.9743e-04
Validation prc-auc = 0.385617
Epoch 10
Validation prc-auc = 0.395383
Epoch 11
Validation prc-auc = 0.333468
Epoch 12
Validation prc-auc = 0.383491
Epoch 13
Validation prc-auc = 0.411578
Epoch 14
Loss = 4.1825e-03, PNorm = 67.0974, GNorm = 1.5012, lr_0 = 9.8890e-04
Validation prc-auc = 0.429017
Epoch 15
Validation prc-auc = 0.463799
Epoch 16
Validation prc-auc = 0.478413
Epoch 17
Validation prc-auc = 0.427881
Epoch 18
Validation prc-auc = 0.347506
Epoch 19
Loss = 4.4951e-03, PNorm = 67.2461, GNorm = 4.0030, lr_0 = 9.8045e-04
Validation prc-auc = 0.331008
Epoch 20
Validation prc-auc = 0.298147
Epoch 21
Validation prc-auc = 0.372987
Epoch 22
Validation prc-auc = 0.424729
Epoch 23
Validation prc-auc = 0.452334
Epoch 24
Loss = 5.4434e-03, PNorm = 67.3827, GNorm = 3.2973, lr_0 = 9.7207e-04
Validation prc-auc = 0.437224
Epoch 25
Validation prc-auc = 0.373108
Epoch 26
Validation prc-auc = 0.257487
Epoch 27
Validation prc-auc = 0.232583
Epoch 28
Validation prc-auc = 0.232410
Epoch 29
Loss = 2.1827e-03, PNorm = 67.4947, GNorm = 1.4135, lr_0 = 9.6376e-04
Validation prc-auc = 0.256107
Epoch 30
Validation prc-auc = 0.270988
Epoch 31
Validation prc-auc = 0.283871
Epoch 32
Validation prc-auc = 0.411748
Epoch 33
Validation prc-auc = 0.428795
Epoch 34
Loss = 1.5135e-03, PNorm = 67.6031, GNorm = 1.3022, lr_0 = 9.5552e-04
Validation prc-auc = 0.435105
Epoch 35
Validation prc-auc = 0.423264
Epoch 36
Validation prc-auc = 0.413432
Epoch 37
Validation prc-auc = 0.411276
Epoch 38
Validation prc-auc = 0.412106
Epoch 39
Loss = 4.5374e-04, PNorm = 67.6851, GNorm = 0.3334, lr_0 = 9.4735e-04
Validation prc-auc = 0.416512
Epoch 40
Validation prc-auc = 0.407345
Epoch 41
Validation prc-auc = 0.416165
Epoch 42
Validation prc-auc = 0.408379
Epoch 43
Validation prc-auc = 0.406699
Epoch 44
Loss = 1.4683e-03, PNorm = 67.7534, GNorm = 0.8439, lr_0 = 9.3925e-04
Validation prc-auc = 0.394915
Epoch 45
Validation prc-auc = 0.384291
Epoch 46
Validation prc-auc = 0.350324
Epoch 47
Validation prc-auc = 0.394849
Epoch 48
Validation prc-auc = 0.398125
Epoch 49
Loss = 1.3105e-03, PNorm = 67.8200, GNorm = 1.5479, lr_0 = 9.3122e-04
Validation prc-auc = 0.417215
Epoch 50
Validation prc-auc = 0.394350
Epoch 51
Validation prc-auc = 0.343750
Epoch 52
Validation prc-auc = 0.278199
Epoch 53
Validation prc-auc = 0.270971
Epoch 54
Loss = 9.8296e-04, PNorm = 67.8868, GNorm = 2.8595, lr_0 = 9.2326e-04
Validation prc-auc = 0.264558
Epoch 55
Validation prc-auc = 0.332920
Epoch 56
Validation prc-auc = 0.395218
Epoch 57
Validation prc-auc = 0.419986
Epoch 58
Validation prc-auc = 0.412714
Epoch 59
Loss = 1.2221e-03, PNorm = 67.9543, GNorm = 1.5211, lr_0 = 9.1537e-04
Validation prc-auc = 0.416442
Epoch 60
Validation prc-auc = 0.413654
Epoch 61
Validation prc-auc = 0.393988
Epoch 62
Validation prc-auc = 0.407488
Epoch 63
Validation prc-auc = 0.409430
Epoch 64
Loss = 8.4661e-04, PNorm = 68.0437, GNorm = 0.7484, lr_0 = 9.0754e-04
Validation prc-auc = 0.399761
Epoch 65
Validation prc-auc = 0.420851
Epoch 66
Validation prc-auc = 0.414452
Epoch 67
Validation prc-auc = 0.373409
Epoch 68
Validation prc-auc = 0.362500
Epoch 69
Loss = 1.7397e-03, PNorm = 68.1327, GNorm = 1.4592, lr_0 = 8.9978e-04
Validation prc-auc = 0.272012
Epoch 70
Validation prc-auc = 0.271137
Epoch 71
Validation prc-auc = 0.348943
Epoch 72
Validation prc-auc = 0.401172
Epoch 73
Validation prc-auc = 0.428192
Epoch 74
Loss = 4.3454e-04, PNorm = 68.2237, GNorm = 0.5317, lr_0 = 8.9209e-04
Validation prc-auc = 0.431874
Epoch 75
Validation prc-auc = 0.418097
Epoch 76
Validation prc-auc = 0.416541
Epoch 77
Validation prc-auc = 0.389170
Epoch 78
Validation prc-auc = 0.370983
Epoch 79
Loss = 9.3269e-04, PNorm = 68.3001, GNorm = 1.5971, lr_0 = 8.8447e-04
Validation prc-auc = 0.362789
Epoch 80
Validation prc-auc = 0.387884
Epoch 81
Validation prc-auc = 0.396910
Epoch 82
Validation prc-auc = 0.407840
Epoch 83
Validation prc-auc = 0.417936
Epoch 84
Loss = 1.6436e-03, PNorm = 68.3787, GNorm = 2.8796, lr_0 = 8.7691e-04
Validation prc-auc = 0.435948
Epoch 85
Validation prc-auc = 0.435355
Epoch 86
Validation prc-auc = 0.434848
Epoch 87
Validation prc-auc = 0.434556
Epoch 88
Validation prc-auc = 0.429622
Epoch 89
Loss = 1.1276e-03, PNorm = 68.4670, GNorm = 0.7155, lr_0 = 8.6941e-04
Validation prc-auc = 0.402591
Epoch 90
Validation prc-auc = 0.371119
Epoch 91
Validation prc-auc = 0.343353
Epoch 92
Validation prc-auc = 0.343937
Epoch 93
Validation prc-auc = 0.363213
Epoch 94
Loss = 5.4063e-04, PNorm = 68.5414, GNorm = 0.5328, lr_0 = 8.6198e-04
Validation prc-auc = 0.401132
Epoch 95
Validation prc-auc = 0.411908
Epoch 96
Validation prc-auc = 0.410662
Epoch 97
Validation prc-auc = 0.378047
Epoch 98
Validation prc-auc = 0.363959
Epoch 99
Loss = 1.3644e-03, PNorm = 68.6192, GNorm = 1.6196, lr_0 = 8.5461e-04
Validation prc-auc = 0.369982
Epoch 100
Validation prc-auc = 0.398530
Epoch 101
Validation prc-auc = 0.430695
Epoch 102
Validation prc-auc = 0.452631
Epoch 103
Validation prc-auc = 0.437571
Epoch 104
Loss = 2.3566e-04, PNorm = 68.7047, GNorm = 0.1780, lr_0 = 8.4730e-04
Validation prc-auc = 0.434068
Epoch 105
Validation prc-auc = 0.422767
Epoch 106
Validation prc-auc = 0.404223
Epoch 107
Validation prc-auc = 0.422288
Epoch 108
Validation prc-auc = 0.430760
Epoch 109
Loss = 6.4610e-04, PNorm = 68.7726, GNorm = 1.5157, lr_0 = 8.4006e-04
Validation prc-auc = 0.414243
Epoch 110
Validation prc-auc = 0.398151
Epoch 111
Validation prc-auc = 0.414507
Epoch 112
Validation prc-auc = 0.434836
Epoch 113
Validation prc-auc = 0.479745
Epoch 114
Loss = 8.5894e-04, PNorm = 68.8572, GNorm = 0.4570, lr_0 = 8.3288e-04
Validation prc-auc = 0.456702
Epoch 115
Validation prc-auc = 0.448574
Epoch 116
Validation prc-auc = 0.456114
Epoch 117
Validation prc-auc = 0.448313
Epoch 118
Validation prc-auc = 0.402701
Epoch 119
Loss = 1.1420e-03, PNorm = 68.9510, GNorm = 1.0839, lr_0 = 8.2576e-04
Validation prc-auc = 0.418013
Epoch 120
Validation prc-auc = 0.416189
Epoch 121
Validation prc-auc = 0.394348
Epoch 122
Validation prc-auc = 0.402369
Epoch 123
Validation prc-auc = 0.387431
Epoch 124
Loss = 5.7571e-03, PNorm = 69.0775, GNorm = 1.5045, lr_0 = 8.1870e-04
Validation prc-auc = 0.332030
Epoch 125
Validation prc-auc = 0.266452
Epoch 126
Validation prc-auc = 0.316167
Epoch 127
Validation prc-auc = 0.344580
Epoch 128
Validation prc-auc = 0.356147
Epoch 129
Loss = 1.9146e-03, PNorm = 69.2369, GNorm = 1.3555, lr_0 = 8.1170e-04
Validation prc-auc = 0.366573
Epoch 130
Validation prc-auc = 0.373147
Epoch 131
Validation prc-auc = 0.377459
Epoch 132
Validation prc-auc = 0.384520
Epoch 133
Validation prc-auc = 0.416716
Epoch 134
Loss = 6.2248e-04, PNorm = 69.3460, GNorm = 1.0415, lr_0 = 8.0476e-04
Validation prc-auc = 0.395163
Epoch 135
Validation prc-auc = 0.364785
Epoch 136
Validation prc-auc = 0.342170
Epoch 137
Validation prc-auc = 0.340576
Epoch 138
Validation prc-auc = 0.339150
Epoch 139
Loss = 8.0370e-05, PNorm = 69.4154, GNorm = 0.0831, lr_0 = 7.9788e-04
Validation prc-auc = 0.344323
Epoch 140
Validation prc-auc = 0.361196
Epoch 141
Validation prc-auc = 0.382621
Epoch 142
Validation prc-auc = 0.383799
Epoch 143
Validation prc-auc = 0.385219
Epoch 144
Loss = 7.4670e-04, PNorm = 69.4558, GNorm = 1.2724, lr_0 = 7.9106e-04
Validation prc-auc = 0.386904
Epoch 145
Validation prc-auc = 0.391832
Epoch 146
Validation prc-auc = 0.396450
Epoch 147
Validation prc-auc = 0.371549
Epoch 148
Validation prc-auc = 0.353720
Epoch 149
Loss = 1.9330e-04, PNorm = 69.5234, GNorm = 0.4229, lr_0 = 7.8430e-04
Validation prc-auc = 0.321142
Epoch 150
Validation prc-auc = 0.318032
Epoch 151
Validation prc-auc = 0.328901
Epoch 152
Validation prc-auc = 0.385843
Epoch 153
Validation prc-auc = 0.428183
Epoch 154
Loss = 1.1655e-04, PNorm = 69.5695, GNorm = 0.2088, lr_0 = 7.7759e-04
Validation prc-auc = 0.434348
Epoch 155
Validation prc-auc = 0.429951
Epoch 156
Validation prc-auc = 0.430573
Epoch 157
Validation prc-auc = 0.395552
Epoch 158
Validation prc-auc = 0.374671
Epoch 159
Loss = 1.0240e-03, PNorm = 69.6296, GNorm = 0.2817, lr_0 = 7.7095e-04
Validation prc-auc = 0.368569
Epoch 160
Validation prc-auc = 0.365658
Epoch 161
Validation prc-auc = 0.368964
Epoch 162
Validation prc-auc = 0.393682
Epoch 163
Validation prc-auc = 0.401455
Epoch 164
Loss = 2.9542e-04, PNorm = 69.6768, GNorm = 0.1474, lr_0 = 7.6436e-04
Validation prc-auc = 0.403203
Epoch 165
Validation prc-auc = 0.409200
Epoch 166
Validation prc-auc = 0.443779
Epoch 167
Validation prc-auc = 0.451088
Epoch 168
Validation prc-auc = 0.446928
Epoch 169
Loss = 2.0813e-04, PNorm = 69.7065, GNorm = 0.7846, lr_0 = 7.5782e-04
Validation prc-auc = 0.440818
Epoch 170
Validation prc-auc = 0.429949
Epoch 171
Validation prc-auc = 0.434782
Epoch 172
Validation prc-auc = 0.409765
Epoch 173
Validation prc-auc = 0.407019
Epoch 174
Loss = 2.3785e-04, PNorm = 69.7434, GNorm = 1.1089, lr_0 = 7.5134e-04
Validation prc-auc = 0.386185
Epoch 175
Validation prc-auc = 0.370534
Epoch 176
Validation prc-auc = 0.373237
Epoch 177
Validation prc-auc = 0.375676
Epoch 178
Validation prc-auc = 0.375973
Epoch 179
Loss = 2.2590e-04, PNorm = 69.7848, GNorm = 0.8803, lr_0 = 7.4492e-04
Validation prc-auc = 0.386620
Epoch 180
Validation prc-auc = 0.387789
Epoch 181
Validation prc-auc = 0.408554
Epoch 182
Validation prc-auc = 0.409995
Epoch 183
Validation prc-auc = 0.390370
Epoch 184
Loss = 4.4957e-05, PNorm = 69.8152, GNorm = 0.2093, lr_0 = 7.3855e-04
Validation prc-auc = 0.392230
Epoch 185
Validation prc-auc = 0.387860
Epoch 186
Validation prc-auc = 0.389529
Epoch 187
Validation prc-auc = 0.390459
Epoch 188
Validation prc-auc = 0.391365
Epoch 189
Loss = 4.7099e-05, PNorm = 69.8339, GNorm = 0.3452, lr_0 = 7.3224e-04
Validation prc-auc = 0.394635
Epoch 190
Validation prc-auc = 0.398259
Epoch 191
Validation prc-auc = 0.425202
Epoch 192
Validation prc-auc = 0.426629
Epoch 193
Validation prc-auc = 0.426838
Epoch 194
Loss = 1.5855e-03, PNorm = 69.8625, GNorm = 2.5945, lr_0 = 7.2598e-04
Validation prc-auc = 0.428609
Epoch 195
Validation prc-auc = 0.416129
Epoch 196
Validation prc-auc = 0.377142
Epoch 197
Validation prc-auc = 0.277503
Epoch 198
Validation prc-auc = 0.281685
Epoch 199
Loss = 1.7659e-03, PNorm = 69.9477, GNorm = 0.3015, lr_0 = 7.1977e-04
Validation prc-auc = 0.303162
Epoch 200
Validation prc-auc = 0.314478
Epoch 201
Validation prc-auc = 0.317152
Epoch 202
Validation prc-auc = 0.331805
Epoch 203
Validation prc-auc = 0.339801
Epoch 204
Loss = 1.4342e-04, PNorm = 70.0682, GNorm = 0.4486, lr_0 = 7.1362e-04
Validation prc-auc = 0.339838
Epoch 205
Validation prc-auc = 0.343697
Epoch 206
Validation prc-auc = 0.385350
Epoch 207
Validation prc-auc = 0.334994
Epoch 208
Validation prc-auc = 0.332139
Epoch 209
Loss = 1.9897e-04, PNorm = 70.1700, GNorm = 0.1863, lr_0 = 7.0752e-04
Validation prc-auc = 0.336474
Epoch 210
Validation prc-auc = 0.336684
Epoch 211
Validation prc-auc = 0.329396
Epoch 212
Validation prc-auc = 0.314544
Epoch 213
Validation prc-auc = 0.309932
Epoch 214
Loss = 2.6473e-04, PNorm = 70.2334, GNorm = 0.1280, lr_0 = 7.0147e-04
Validation prc-auc = 0.306463
Epoch 215
Validation prc-auc = 0.291925
Epoch 216
Validation prc-auc = 0.281347
Epoch 217
Validation prc-auc = 0.281998
Epoch 218
Validation prc-auc = 0.361101
Epoch 219
Loss = 2.5696e-05, PNorm = 70.2778, GNorm = 0.0265, lr_0 = 6.9548e-04
Validation prc-auc = 0.402498
Epoch 220
Validation prc-auc = 0.420940
Epoch 221
Validation prc-auc = 0.425915
Epoch 222
Validation prc-auc = 0.435083
Epoch 223
Validation prc-auc = 0.433372
Epoch 224
Loss = 2.5217e-05, PNorm = 70.3152, GNorm = 0.0406, lr_0 = 6.8953e-04
Validation prc-auc = 0.414768
Epoch 225
Validation prc-auc = 0.407107
Epoch 226
Validation prc-auc = 0.401260
Epoch 227
Validation prc-auc = 0.403039
Epoch 228
Validation prc-auc = 0.405135
Epoch 229
Loss = 6.8368e-05, PNorm = 70.3441, GNorm = 0.2079, lr_0 = 6.8364e-04
Validation prc-auc = 0.388120
Epoch 230
Validation prc-auc = 0.390347
Epoch 231
Validation prc-auc = 0.392333
Epoch 232
Validation prc-auc = 0.379019
Epoch 233
Validation prc-auc = 0.346176
Epoch 234
Loss = 1.1452e-04, PNorm = 70.3714, GNorm = 0.4778, lr_0 = 6.7779e-04
Validation prc-auc = 0.336770
Epoch 235
Validation prc-auc = 0.278155
Epoch 236
Validation prc-auc = 0.282627
Epoch 237
Validation prc-auc = 0.347818
Epoch 238
Validation prc-auc = 0.357713
Epoch 239
Loss = 8.6156e-05, PNorm = 70.3976, GNorm = 0.1553, lr_0 = 6.7200e-04
Validation prc-auc = 0.388093
Epoch 240
Validation prc-auc = 0.386710
Epoch 241
Validation prc-auc = 0.359845
Epoch 242
Validation prc-auc = 0.354465
Epoch 243
Validation prc-auc = 0.299255
Epoch 244
Loss = 9.6862e-06, PNorm = 70.4205, GNorm = 0.0080, lr_0 = 6.6625e-04
Validation prc-auc = 0.288169
Epoch 245
Validation prc-auc = 0.288144
Epoch 246
Validation prc-auc = 0.284454
Epoch 247
Validation prc-auc = 0.287742
Epoch 248
Validation prc-auc = 0.290579
Epoch 249
Loss = 3.6973e-05, PNorm = 70.4462, GNorm = 0.0187, lr_0 = 6.6056e-04
Validation prc-auc = 0.290826
Epoch 250
Validation prc-auc = 0.288304
Epoch 251
Validation prc-auc = 0.288691
Epoch 252
Validation prc-auc = 0.289253
Epoch 253
Validation prc-auc = 0.338042
Epoch 254
Loss = 4.1370e-05, PNorm = 70.4679, GNorm = 0.0475, lr_0 = 6.5491e-04
Validation prc-auc = 0.346054
Epoch 255
Validation prc-auc = 0.350573
Epoch 256
Validation prc-auc = 0.351989
Epoch 257
Validation prc-auc = 0.387487
Epoch 258
Validation prc-auc = 0.387900
Epoch 259
Loss = 7.9507e-05, PNorm = 70.4827, GNorm = 0.3462, lr_0 = 6.4931e-04
Validation prc-auc = 0.385786
Epoch 260
Validation prc-auc = 0.385802
Epoch 261
Validation prc-auc = 0.383659
Epoch 262
Validation prc-auc = 0.384743
Epoch 263
Validation prc-auc = 0.382187
Epoch 264
Loss = 3.6531e-04, PNorm = 70.4923, GNorm = 0.1362, lr_0 = 6.4376e-04
Validation prc-auc = 0.381508
Epoch 265
Validation prc-auc = 0.384756
Epoch 266
Validation prc-auc = 0.386539
Epoch 267
Validation prc-auc = 0.386076
Epoch 268
Validation prc-auc = 0.382988
Epoch 269
Loss = 4.8557e-05, PNorm = 70.5077, GNorm = 0.1175, lr_0 = 6.3826e-04
Validation prc-auc = 0.373708
Epoch 270
Validation prc-auc = 0.370970
Epoch 271
Validation prc-auc = 0.371020
Epoch 272
Validation prc-auc = 0.373092
Epoch 273
Validation prc-auc = 0.375973
Epoch 274
Loss = 3.8595e-05, PNorm = 70.5219, GNorm = 0.2465, lr_0 = 6.3280e-04
Validation prc-auc = 0.379386
Epoch 275
Validation prc-auc = 0.380931
Epoch 276
Validation prc-auc = 0.368147
Epoch 277
Validation prc-auc = 0.367035
Epoch 278
Validation prc-auc = 0.366630
Epoch 279
Loss = 6.4256e-06, PNorm = 70.5292, GNorm = 0.0175, lr_0 = 6.2739e-04
Validation prc-auc = 0.366018
Epoch 280
Validation prc-auc = 0.354439
Epoch 281
Validation prc-auc = 0.354704
Epoch 282
Validation prc-auc = 0.358052
Epoch 283
Validation prc-auc = 0.358724
Epoch 284
Loss = 7.1786e-04, PNorm = 70.5398, GNorm = 1.7998, lr_0 = 6.2203e-04
Validation prc-auc = 0.345390
Epoch 285
Validation prc-auc = 0.343526
Epoch 286
Validation prc-auc = 0.350620
Epoch 287
Validation prc-auc = 0.394729
Epoch 288
Validation prc-auc = 0.402960
Epoch 289
Loss = 2.8625e-05, PNorm = 70.5676, GNorm = 0.0545, lr_0 = 6.1671e-04
Validation prc-auc = 0.418429
Epoch 290
Validation prc-auc = 0.417292
Epoch 291
Validation prc-auc = 0.404301
Epoch 292
Validation prc-auc = 0.365187
Epoch 293
Validation prc-auc = 0.364194
Epoch 294
Loss = 5.7134e-06, PNorm = 70.6017, GNorm = 0.0071, lr_0 = 6.1144e-04
Validation prc-auc = 0.353184
Epoch 295
Validation prc-auc = 0.295151
Epoch 296
Validation prc-auc = 0.302956
Epoch 297
Validation prc-auc = 0.285972
Epoch 298
Validation prc-auc = 0.287683
Epoch 299
Loss = 2.3244e-04, PNorm = 70.6238, GNorm = 0.2479, lr_0 = 6.0621e-04
Validation prc-auc = 0.281288
Model 0 best validation prc-auc = 0.481082 on epoch 0
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.443862
Ensemble test prc-auc = 0.443862
Fold 4
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_4',
 'save_smiles_splits': False,
 'seed': 4,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.413983
Best score so far is 0.413983 at epoch 0 
Epoch 1
Validation prc-auc = 0.421706
Best score so far is 0.421706 at epoch 1 
Epoch 2
Validation prc-auc = 0.385258
Epoch 3
Validation prc-auc = 0.349878
Epoch 4
Loss = 1.0693e-02, PNorm = 66.7732, GNorm = 2.1925, lr_0 = 6.5000e-04
Validation prc-auc = 0.368341
Epoch 5
Validation prc-auc = 0.362120
Epoch 6
Validation prc-auc = 0.381913
Epoch 7
Validation prc-auc = 0.422003
Best score so far is 0.422003 at epoch 7 
Epoch 8
Validation prc-auc = 0.435869
Best score so far is 0.435869 at epoch 8 
Epoch 9
Loss = 5.1701e-03, PNorm = 66.9169, GNorm = 1.7966, lr_0 = 9.9743e-04
Validation prc-auc = 0.449380
Best score so far is 0.449380 at epoch 9 
Epoch 10
Validation prc-auc = 0.462190
Best score so far is 0.462190 at epoch 10 
Epoch 11
Validation prc-auc = 0.414611
Epoch 12
Validation prc-auc = 0.324615
Epoch 13
Validation prc-auc = 0.308445
Epoch 14
Loss = 4.2649e-03, PNorm = 67.0988, GNorm = 1.2181, lr_0 = 9.8890e-04
Validation prc-auc = 0.313520
Epoch 15
Validation prc-auc = 0.349011
Epoch 16
Validation prc-auc = 0.385900
Epoch 17
Validation prc-auc = 0.381693
Epoch 18
Validation prc-auc = 0.352106
Epoch 19
Loss = 3.4819e-03, PNorm = 67.2486, GNorm = 1.6007, lr_0 = 9.8045e-04
Validation prc-auc = 0.293725
Epoch 20
Validation prc-auc = 0.298354
Epoch 21
Validation prc-auc = 0.315924
Epoch 22
Validation prc-auc = 0.361765
Epoch 23
Validation prc-auc = 0.366396
Epoch 24
Loss = 3.2703e-03, PNorm = 67.3678, GNorm = 0.7240, lr_0 = 9.7207e-04
Validation prc-auc = 0.351864
Epoch 25
Validation prc-auc = 0.340260
Epoch 26
Validation prc-auc = 0.306554
Epoch 27
Validation prc-auc = 0.338672
Epoch 28
Validation prc-auc = 0.347217
Epoch 29
Loss = 2.9285e-03, PNorm = 67.4685, GNorm = 1.3111, lr_0 = 9.6376e-04
Validation prc-auc = 0.339653
Epoch 30
Validation prc-auc = 0.334852
Epoch 31
Validation prc-auc = 0.348988
Epoch 32
Validation prc-auc = 0.366080
Epoch 33
Validation prc-auc = 0.372432
Epoch 34
Loss = 2.2736e-03, PNorm = 67.5761, GNorm = 1.6911, lr_0 = 9.5552e-04
Validation prc-auc = 0.390086
Epoch 35
Validation prc-auc = 0.385710
Epoch 36
Validation prc-auc = 0.369656
Epoch 37
Validation prc-auc = 0.342414
Epoch 38
Validation prc-auc = 0.305786
Epoch 39
Loss = 1.6910e-03, PNorm = 67.6851, GNorm = 1.0191, lr_0 = 9.4735e-04
Validation prc-auc = 0.318733
Epoch 40
Validation prc-auc = 0.360524
Epoch 41
Validation prc-auc = 0.298890
Epoch 42
Validation prc-auc = 0.304142
Epoch 43
Validation prc-auc = 0.297216
Epoch 44
Loss = 2.2172e-03, PNorm = 67.7842, GNorm = 2.1538, lr_0 = 9.3925e-04
Validation prc-auc = 0.285400
Epoch 45
Validation prc-auc = 0.402826
Epoch 46
Validation prc-auc = 0.393654
Epoch 47
Validation prc-auc = 0.413243
Epoch 48
Validation prc-auc = 0.383578
Epoch 49
Loss = 2.0428e-03, PNorm = 67.8746, GNorm = 1.7572, lr_0 = 9.3122e-04
Validation prc-auc = 0.295815
Epoch 50
Validation prc-auc = 0.233388
Epoch 51
Validation prc-auc = 0.253756
Epoch 52
Validation prc-auc = 0.304305
Epoch 53
Validation prc-auc = 0.423099
Epoch 54
Loss = 2.0379e-03, PNorm = 67.9551, GNorm = 1.7913, lr_0 = 9.2326e-04
Validation prc-auc = 0.456122
Epoch 55
Validation prc-auc = 0.462541
Best score so far is 0.462541 at epoch 55 
Epoch 56
Validation prc-auc = 0.373645
Epoch 57
Validation prc-auc = 0.339009
Epoch 58
Validation prc-auc = 0.325672
Epoch 59
Loss = 1.5104e-03, PNorm = 68.0506, GNorm = 1.7439, lr_0 = 9.1537e-04
Validation prc-auc = 0.322790
Epoch 60
Validation prc-auc = 0.332796
Epoch 61
Validation prc-auc = 0.346529
Epoch 62
Validation prc-auc = 0.352718
Epoch 63
Validation prc-auc = 0.370666
Epoch 64
Loss = 5.7400e-04, PNorm = 68.1316, GNorm = 1.1802, lr_0 = 9.0754e-04
Validation prc-auc = 0.369684
Epoch 65
Validation prc-auc = 0.395802
Epoch 66
Validation prc-auc = 0.399359
Epoch 67
Validation prc-auc = 0.405970
Epoch 68
Validation prc-auc = 0.389397
Epoch 69
Loss = 4.0263e-04, PNorm = 68.2042, GNorm = 0.6170, lr_0 = 8.9978e-04
Validation prc-auc = 0.391201
Epoch 70
Validation prc-auc = 0.367238
Epoch 71
Validation prc-auc = 0.340631
Epoch 72
Validation prc-auc = 0.335143
Epoch 73
Validation prc-auc = 0.342743
Epoch 74
Loss = 1.3415e-03, PNorm = 68.2761, GNorm = 2.4046, lr_0 = 8.9209e-04
Validation prc-auc = 0.329872
Epoch 75
Validation prc-auc = 0.341357
Epoch 76
Validation prc-auc = 0.340714
Epoch 77
Validation prc-auc = 0.337127
Epoch 78
Validation prc-auc = 0.328168
Epoch 79
Loss = 4.6505e-04, PNorm = 68.3536, GNorm = 0.4906, lr_0 = 8.8447e-04
Validation prc-auc = 0.317440
Epoch 80
Validation prc-auc = 0.293812
Epoch 81
Validation prc-auc = 0.246383
Epoch 82
Validation prc-auc = 0.256265
Epoch 83
Validation prc-auc = 0.344405
Epoch 84
Loss = 1.7947e-03, PNorm = 68.4417, GNorm = 1.8324, lr_0 = 8.7691e-04
Validation prc-auc = 0.370310
Epoch 85
Validation prc-auc = 0.375640
Epoch 86
Validation prc-auc = 0.379498
Epoch 87
Validation prc-auc = 0.370628
Epoch 88
Validation prc-auc = 0.367227
Epoch 89
Loss = 3.8526e-04, PNorm = 68.5062, GNorm = 0.2265, lr_0 = 8.6941e-04
Validation prc-auc = 0.368614
Epoch 90
Validation prc-auc = 0.355383
Epoch 91
Validation prc-auc = 0.347623
Epoch 92
Validation prc-auc = 0.344850
Epoch 93
Validation prc-auc = 0.315217
Epoch 94
Loss = 6.4540e-04, PNorm = 68.5848, GNorm = 1.9583, lr_0 = 8.6198e-04
Validation prc-auc = 0.344785
Epoch 95
Validation prc-auc = 0.360551
Epoch 96
Validation prc-auc = 0.366345
Epoch 97
Validation prc-auc = 0.371304
Epoch 98
Validation prc-auc = 0.368784
Epoch 99
Loss = 3.1784e-04, PNorm = 68.6537, GNorm = 0.0793, lr_0 = 8.5461e-04
Validation prc-auc = 0.377207
Epoch 100
Validation prc-auc = 0.374861
Epoch 101
Validation prc-auc = 0.377613
Epoch 102
Validation prc-auc = 0.387499
Epoch 103
Validation prc-auc = 0.371449
Epoch 104
Loss = 7.2583e-04, PNorm = 68.7157, GNorm = 1.8644, lr_0 = 8.4730e-04
Validation prc-auc = 0.379941
Epoch 105
Validation prc-auc = 0.352649
Epoch 106
Validation prc-auc = 0.356970
Epoch 107
Validation prc-auc = 0.404083
Epoch 108
Validation prc-auc = 0.414991
Epoch 109
Loss = 2.9071e-03, PNorm = 68.7996, GNorm = 2.4723, lr_0 = 8.4006e-04
Validation prc-auc = 0.395112
Epoch 110
Validation prc-auc = 0.304398
Epoch 111
Validation prc-auc = 0.281851
Epoch 112
Validation prc-auc = 0.258523
Epoch 113
Validation prc-auc = 0.284354
Epoch 114
Loss = 1.0422e-03, PNorm = 68.8946, GNorm = 0.4238, lr_0 = 8.3288e-04
Validation prc-auc = 0.373196
Epoch 115
Validation prc-auc = 0.433957
Epoch 116
Validation prc-auc = 0.427563
Epoch 117
Validation prc-auc = 0.423526
Epoch 118
Validation prc-auc = 0.422697
Epoch 119
Loss = 1.2560e-03, PNorm = 69.0025, GNorm = 1.0655, lr_0 = 8.2576e-04
Validation prc-auc = 0.396959
Epoch 120
Validation prc-auc = 0.371568
Epoch 121
Validation prc-auc = 0.350216
Epoch 122
Validation prc-auc = 0.231628
Epoch 123
Validation prc-auc = 0.219445
Epoch 124
Loss = 3.0701e-04, PNorm = 69.1079, GNorm = 0.7888, lr_0 = 8.1870e-04
Validation prc-auc = 0.213083
Epoch 125
Validation prc-auc = 0.243308
Epoch 126
Validation prc-auc = 0.260308
Epoch 127
Validation prc-auc = 0.268067
Epoch 128
Validation prc-auc = 0.289330
Epoch 129
Loss = 7.4195e-04, PNorm = 69.2013, GNorm = 0.0987, lr_0 = 8.1170e-04
Validation prc-auc = 0.285275
Epoch 130
Validation prc-auc = 0.287195
Epoch 131
Validation prc-auc = 0.281814
Epoch 132
Validation prc-auc = 0.370670
Epoch 133
Validation prc-auc = 0.394185
Epoch 134
Loss = 4.6568e-04, PNorm = 69.3126, GNorm = 0.5858, lr_0 = 8.0476e-04
Validation prc-auc = 0.395102
Epoch 135
Validation prc-auc = 0.391713
Epoch 136
Validation prc-auc = 0.381245
Epoch 137
Validation prc-auc = 0.372964
Epoch 138
Validation prc-auc = 0.371427
Epoch 139
Loss = 9.2315e-05, PNorm = 69.3814, GNorm = 0.0808, lr_0 = 7.9788e-04
Validation prc-auc = 0.368814
Epoch 140
Validation prc-auc = 0.378940
Epoch 141
Validation prc-auc = 0.372004
Epoch 142
Validation prc-auc = 0.389882
Epoch 143
Validation prc-auc = 0.385685
Epoch 144
Loss = 7.4641e-05, PNorm = 69.4415, GNorm = 0.2432, lr_0 = 7.9106e-04
Validation prc-auc = 0.386525
Epoch 145
Validation prc-auc = 0.381107
Epoch 146
Validation prc-auc = 0.381260
Epoch 147
Validation prc-auc = 0.378591
Epoch 148
Validation prc-auc = 0.378461
Epoch 149
Loss = 3.6502e-04, PNorm = 69.5417, GNorm = 1.4191, lr_0 = 7.8430e-04
Validation prc-auc = 0.376864
Epoch 150
Validation prc-auc = 0.381208
Epoch 151
Validation prc-auc = 0.383704
Epoch 152
Validation prc-auc = 0.391714
Epoch 153
Validation prc-auc = 0.412172
Epoch 154
Loss = 9.6389e-04, PNorm = 69.6030, GNorm = 2.3327, lr_0 = 7.7759e-04
Validation prc-auc = 0.407102
Epoch 155
Validation prc-auc = 0.391960
Epoch 156
Validation prc-auc = 0.360128
Epoch 157
Validation prc-auc = 0.316747
Epoch 158
Validation prc-auc = 0.243159
Epoch 159
Loss = 4.3240e-04, PNorm = 69.6758, GNorm = 0.4609, lr_0 = 7.7095e-04
Validation prc-auc = 0.235741
Epoch 160
Validation prc-auc = 0.237202
Epoch 161
Validation prc-auc = 0.305866
Epoch 162
Validation prc-auc = 0.347888
Epoch 163
Validation prc-auc = 0.371947
Epoch 164
Loss = 3.5276e-04, PNorm = 69.7680, GNorm = 0.9896, lr_0 = 7.6436e-04
Validation prc-auc = 0.382822
Epoch 165
Validation prc-auc = 0.381874
Epoch 166
Validation prc-auc = 0.379218
Epoch 167
Validation prc-auc = 0.370777
Epoch 168
Validation prc-auc = 0.358272
Epoch 169
Loss = 3.5556e-05, PNorm = 69.8387, GNorm = 0.0788, lr_0 = 7.5782e-04
Validation prc-auc = 0.347692
Epoch 170
Validation prc-auc = 0.343550
Epoch 171
Validation prc-auc = 0.343691
Epoch 172
Validation prc-auc = 0.360255
Epoch 173
Validation prc-auc = 0.365986
Epoch 174
Loss = 2.6617e-04, PNorm = 69.8852, GNorm = 0.1274, lr_0 = 7.5134e-04
Validation prc-auc = 0.376313
Epoch 175
Validation prc-auc = 0.394676
Epoch 176
Validation prc-auc = 0.422256
Epoch 177
Validation prc-auc = 0.421283
Epoch 178
Validation prc-auc = 0.425861
Epoch 179
Loss = 1.0724e-03, PNorm = 69.9501, GNorm = 2.0449, lr_0 = 7.4492e-04
Validation prc-auc = 0.397381
Epoch 180
Validation prc-auc = 0.385777
Epoch 181
Validation prc-auc = 0.369902
Epoch 182
Validation prc-auc = 0.328389
Epoch 183
Validation prc-auc = 0.270442
Epoch 184
Loss = 9.2472e-04, PNorm = 70.0213, GNorm = 0.5343, lr_0 = 7.3855e-04
Validation prc-auc = 0.266066
Epoch 185
Validation prc-auc = 0.314491
Epoch 186
Validation prc-auc = 0.351966
Epoch 187
Validation prc-auc = 0.376482
Epoch 188
Validation prc-auc = 0.391862
Epoch 189
Loss = 1.2834e-04, PNorm = 70.0838, GNorm = 0.0124, lr_0 = 7.3224e-04
Validation prc-auc = 0.411861
Epoch 190
Validation prc-auc = 0.415055
Epoch 191
Validation prc-auc = 0.422516
Epoch 192
Validation prc-auc = 0.430204
Epoch 193
Validation prc-auc = 0.431388
Epoch 194
Loss = 5.8288e-04, PNorm = 70.1327, GNorm = 0.0263, lr_0 = 7.2598e-04
Validation prc-auc = 0.436908
Epoch 195
Validation prc-auc = 0.422615
Epoch 196
Validation prc-auc = 0.406809
Epoch 197
Validation prc-auc = 0.399684
Epoch 198
Validation prc-auc = 0.393344
Epoch 199
Loss = 8.3056e-04, PNorm = 70.1635, GNorm = 0.2830, lr_0 = 7.1977e-04
Validation prc-auc = 0.387530
Epoch 200
Validation prc-auc = 0.382150
Epoch 201
Validation prc-auc = 0.378303
Epoch 202
Validation prc-auc = 0.381447
Epoch 203
Validation prc-auc = 0.387492
Epoch 204
Loss = 3.6699e-04, PNorm = 70.2019, GNorm = 0.0056, lr_0 = 7.1362e-04
Validation prc-auc = 0.386488
Epoch 205
Validation prc-auc = 0.397751
Epoch 206
Validation prc-auc = 0.405568
Epoch 207
Validation prc-auc = 0.411020
Epoch 208
Validation prc-auc = 0.421387
Epoch 209
Loss = 1.6453e-04, PNorm = 70.2535, GNorm = 0.1313, lr_0 = 7.0752e-04
Validation prc-auc = 0.419356
Epoch 210
Validation prc-auc = 0.418655
Epoch 211
Validation prc-auc = 0.410934
Epoch 212
Validation prc-auc = 0.403612
Epoch 213
Validation prc-auc = 0.405559
Epoch 214
Loss = 2.0408e-04, PNorm = 70.3219, GNorm = 0.7599, lr_0 = 7.0147e-04
Validation prc-auc = 0.394004
Epoch 215
Validation prc-auc = 0.397121
Epoch 216
Validation prc-auc = 0.401305
Epoch 217
Validation prc-auc = 0.414990
Epoch 218
Validation prc-auc = 0.421181
Epoch 219
Loss = 3.4120e-05, PNorm = 70.3479, GNorm = 0.0438, lr_0 = 6.9548e-04
Validation prc-auc = 0.428854
Epoch 220
Validation prc-auc = 0.429558
Epoch 221
Validation prc-auc = 0.427380
Epoch 222
Validation prc-auc = 0.419626
Epoch 223
Validation prc-auc = 0.404041
Epoch 224
Loss = 2.1313e-05, PNorm = 70.3713, GNorm = 0.0457, lr_0 = 6.8953e-04
Validation prc-auc = 0.400010
Epoch 225
Validation prc-auc = 0.380220
Epoch 226
Validation prc-auc = 0.364073
Epoch 227
Validation prc-auc = 0.358433
Epoch 228
Validation prc-auc = 0.356401
Epoch 229
Loss = 9.0624e-05, PNorm = 70.3807, GNorm = 0.5807, lr_0 = 6.8364e-04
Validation prc-auc = 0.355611
Epoch 230
Validation prc-auc = 0.356916
Epoch 231
Validation prc-auc = 0.356715
Epoch 232
Validation prc-auc = 0.359091
Epoch 233
Validation prc-auc = 0.366861
Epoch 234
Loss = 3.5097e-05, PNorm = 70.3953, GNorm = 0.0666, lr_0 = 6.7779e-04
Validation prc-auc = 0.368513
Epoch 235
Validation prc-auc = 0.376967
Epoch 236
Validation prc-auc = 0.382420
Epoch 237
Validation prc-auc = 0.385778
Epoch 238
Validation prc-auc = 0.388089
Epoch 239
Loss = 1.2792e-04, PNorm = 70.4199, GNorm = 0.0149, lr_0 = 6.7200e-04
Validation prc-auc = 0.396453
Epoch 240
Validation prc-auc = 0.403795
Epoch 241
Validation prc-auc = 0.411562
Epoch 242
Validation prc-auc = 0.412004
Epoch 243
Validation prc-auc = 0.413604
Epoch 244
Loss = 2.2612e-05, PNorm = 70.4450, GNorm = 0.0582, lr_0 = 6.6625e-04
Validation prc-auc = 0.412670
Epoch 245
Validation prc-auc = 0.414512
Epoch 246
Validation prc-auc = 0.415544
Epoch 247
Validation prc-auc = 0.416367
Epoch 248
Validation prc-auc = 0.420644
Epoch 249
Loss = 5.9252e-04, PNorm = 70.4534, GNorm = 2.3892, lr_0 = 6.6056e-04
Validation prc-auc = 0.407124
Epoch 250
Validation prc-auc = 0.394867
Epoch 251
Validation prc-auc = 0.315646
Epoch 252
Validation prc-auc = 0.307265
Epoch 253
Validation prc-auc = 0.305353
Epoch 254
Loss = 2.0393e-04, PNorm = 70.4834, GNorm = 1.1813, lr_0 = 6.5491e-04
Validation prc-auc = 0.303360
Epoch 255
Validation prc-auc = 0.301624
Epoch 256
Validation prc-auc = 0.304893
Epoch 257
Validation prc-auc = 0.307831
Epoch 258
Validation prc-auc = 0.308397
Epoch 259
Loss = 1.0073e-04, PNorm = 70.5212, GNorm = 0.1896, lr_0 = 6.4931e-04
Validation prc-auc = 0.300760
Epoch 260
Validation prc-auc = 0.378875
Epoch 261
Validation prc-auc = 0.374674
Epoch 262
Validation prc-auc = 0.379964
Epoch 263
Validation prc-auc = 0.387332
Epoch 264
Loss = 1.6617e-04, PNorm = 70.5619, GNorm = 0.0171, lr_0 = 6.4376e-04
Validation prc-auc = 0.400752
Epoch 265
Validation prc-auc = 0.407967
Epoch 266
Validation prc-auc = 0.409959
Epoch 267
Validation prc-auc = 0.405071
Epoch 268
Validation prc-auc = 0.393845
Epoch 269
Loss = 1.7067e-05, PNorm = 70.5987, GNorm = 0.0801, lr_0 = 6.3826e-04
Validation prc-auc = 0.390259
Epoch 270
Validation prc-auc = 0.388899
Epoch 271
Validation prc-auc = 0.390701
Epoch 272
Validation prc-auc = 0.390233
Epoch 273
Validation prc-auc = 0.391522
Epoch 274
Loss = 1.4780e-05, PNorm = 70.6432, GNorm = 0.0706, lr_0 = 6.3280e-04
Validation prc-auc = 0.395759
Epoch 275
Validation prc-auc = 0.394236
Epoch 276
Validation prc-auc = 0.391259
Epoch 277
Validation prc-auc = 0.386755
Epoch 278
Validation prc-auc = 0.388358
Epoch 279
Loss = 1.1654e-05, PNorm = 70.6753, GNorm = 0.0288, lr_0 = 6.2739e-04
Validation prc-auc = 0.389005
Epoch 280
Validation prc-auc = 0.380728
Epoch 281
Validation prc-auc = 0.388260
Epoch 282
Validation prc-auc = 0.396050
Epoch 283
Validation prc-auc = 0.404036
Epoch 284
Loss = 2.4274e-05, PNorm = 70.6933, GNorm = 0.1319, lr_0 = 6.2203e-04
Validation prc-auc = 0.410015
Epoch 285
Validation prc-auc = 0.409957
Epoch 286
Validation prc-auc = 0.405877
Epoch 287
Validation prc-auc = 0.401773
Epoch 288
Validation prc-auc = 0.400502
Epoch 289
Loss = 5.1419e-06, PNorm = 70.7080, GNorm = 0.0040, lr_0 = 6.1671e-04
Validation prc-auc = 0.399370
Epoch 290
Validation prc-auc = 0.398831
Epoch 291
Validation prc-auc = 0.397970
Epoch 292
Validation prc-auc = 0.395900
Epoch 293
Validation prc-auc = 0.392711
Epoch 294
Loss = 8.0638e-06, PNorm = 70.7175, GNorm = 0.0430, lr_0 = 6.1144e-04
Validation prc-auc = 0.392852
Epoch 295
Validation prc-auc = 0.391101
Epoch 296
Validation prc-auc = 0.389284
Epoch 297
Validation prc-auc = 0.388955
Epoch 298
Validation prc-auc = 0.379765
Epoch 299
Loss = 3.2916e-06, PNorm = 70.7275, GNorm = 0.0059, lr_0 = 6.0621e-04
Validation prc-auc = 0.374965
Model 0 best validation prc-auc = 0.462541 on epoch 55
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.416158
Ensemble test prc-auc = 0.416158
Fold 5
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_5',
 'save_smiles_splits': False,
 'seed': 5,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.443447
Best score so far is 0.443447 at epoch 0 
Epoch 1
Validation prc-auc = 0.427204
Epoch 2
Validation prc-auc = 0.383075
Epoch 3
Validation prc-auc = 0.373791
Epoch 4
Loss = 1.0490e-02, PNorm = 66.7744, GNorm = 3.2465, lr_0 = 6.5000e-04
Validation prc-auc = 0.362358
Epoch 5
Validation prc-auc = 0.365211
Epoch 6
Validation prc-auc = 0.349751
Epoch 7
Validation prc-auc = 0.341198
Epoch 8
Validation prc-auc = 0.323456
Epoch 9
Loss = 5.6723e-03, PNorm = 66.9289, GNorm = 1.4486, lr_0 = 9.9743e-04
Validation prc-auc = 0.361887
Epoch 10
Validation prc-auc = 0.338805
Epoch 11
Validation prc-auc = 0.311553
Epoch 12
Validation prc-auc = 0.265818
Epoch 13
Validation prc-auc = 0.242099
Epoch 14
Loss = 7.6223e-03, PNorm = 67.1055, GNorm = 4.8502, lr_0 = 9.8890e-04
Validation prc-auc = 0.285200
Epoch 15
Validation prc-auc = 0.351448
Epoch 16
Validation prc-auc = 0.393020
Epoch 17
Validation prc-auc = 0.287425
Epoch 18
Validation prc-auc = 0.370682
Epoch 19
Loss = 4.1396e-03, PNorm = 67.2600, GNorm = 2.1375, lr_0 = 9.8045e-04
Validation prc-auc = 0.385982
Epoch 20
Validation prc-auc = 0.387426
Epoch 21
Validation prc-auc = 0.365013
Epoch 22
Validation prc-auc = 0.353583
Epoch 23
Validation prc-auc = 0.353049
Epoch 24
Loss = 4.2282e-03, PNorm = 67.3886, GNorm = 1.4340, lr_0 = 9.7207e-04
Validation prc-auc = 0.363483
Epoch 25
Validation prc-auc = 0.359217
Epoch 26
Validation prc-auc = 0.366538
Epoch 27
Validation prc-auc = 0.327645
Epoch 28
Validation prc-auc = 0.348244
Epoch 29
Loss = 1.6984e-03, PNorm = 67.5063, GNorm = 1.1259, lr_0 = 9.6376e-04
Validation prc-auc = 0.358474
Epoch 30
Validation prc-auc = 0.380452
Epoch 31
Validation prc-auc = 0.377160
Epoch 32
Validation prc-auc = 0.361875
Epoch 33
Validation prc-auc = 0.323595
Epoch 34
Loss = 1.5627e-03, PNorm = 67.6092, GNorm = 0.9200, lr_0 = 9.5552e-04
Validation prc-auc = 0.347673
Epoch 35
Validation prc-auc = 0.308301
Epoch 36
Validation prc-auc = 0.326325
Epoch 37
Validation prc-auc = 0.351384
Epoch 38
Validation prc-auc = 0.378281
Epoch 39
Loss = 3.2455e-03, PNorm = 67.7186, GNorm = 1.5682, lr_0 = 9.4735e-04
Validation prc-auc = 0.381468
Epoch 40
Validation prc-auc = 0.388134
Epoch 41
Validation prc-auc = 0.380705
Epoch 42
Validation prc-auc = 0.332853
Epoch 43
Validation prc-auc = 0.269135
Epoch 44
Loss = 2.7125e-03, PNorm = 67.8204, GNorm = 1.4647, lr_0 = 9.3925e-04
Validation prc-auc = 0.305837
Epoch 45
Validation prc-auc = 0.328708
Epoch 46
Validation prc-auc = 0.359893
Epoch 47
Validation prc-auc = 0.363597
Epoch 48
Validation prc-auc = 0.360587
Epoch 49
Loss = 8.0159e-04, PNorm = 67.9082, GNorm = 1.3897, lr_0 = 9.3122e-04
Validation prc-auc = 0.353255
Epoch 50
Validation prc-auc = 0.348128
Epoch 51
Validation prc-auc = 0.329295
Epoch 52
Validation prc-auc = 0.313597
Epoch 53
Validation prc-auc = 0.306598
Epoch 54
Loss = 1.0892e-03, PNorm = 67.9834, GNorm = 2.2165, lr_0 = 9.2326e-04
Validation prc-auc = 0.305368
Epoch 55
Validation prc-auc = 0.331954
Epoch 56
Validation prc-auc = 0.393247
Epoch 57
Validation prc-auc = 0.431201
Epoch 58
Validation prc-auc = 0.432075
Epoch 59
Loss = 2.4386e-04, PNorm = 68.0662, GNorm = 0.3488, lr_0 = 9.1537e-04
Validation prc-auc = 0.427258
Epoch 60
Validation prc-auc = 0.421869
Epoch 61
Validation prc-auc = 0.407861
Epoch 62
Validation prc-auc = 0.350886
Epoch 63
Validation prc-auc = 0.249376
Epoch 64
Loss = 1.6488e-03, PNorm = 68.1529, GNorm = 3.4228, lr_0 = 9.0754e-04
Validation prc-auc = 0.247188
Epoch 65
Validation prc-auc = 0.307197
Epoch 66
Validation prc-auc = 0.370735
Epoch 67
Validation prc-auc = 0.375400
Epoch 68
Validation prc-auc = 0.367467
Epoch 69
Loss = 2.1449e-03, PNorm = 68.2309, GNorm = 2.0002, lr_0 = 8.9978e-04
Validation prc-auc = 0.345834
Epoch 70
Validation prc-auc = 0.238320
Epoch 71
Validation prc-auc = 0.317910
Epoch 72
Validation prc-auc = 0.315613
Epoch 73
Validation prc-auc = 0.362549
Epoch 74
Loss = 7.2205e-04, PNorm = 68.3240, GNorm = 0.5804, lr_0 = 8.9209e-04
Validation prc-auc = 0.364377
Epoch 75
Validation prc-auc = 0.367194
Epoch 76
Validation prc-auc = 0.381478
Epoch 77
Validation prc-auc = 0.374638
Epoch 78
Validation prc-auc = 0.372847
Epoch 79
Loss = 5.2519e-04, PNorm = 68.4027, GNorm = 1.1158, lr_0 = 8.8447e-04
Validation prc-auc = 0.355966
Epoch 80
Validation prc-auc = 0.346897
Epoch 81
Validation prc-auc = 0.356362
Epoch 82
Validation prc-auc = 0.338154
Epoch 83
Validation prc-auc = 0.340696
Epoch 84
Loss = 1.3186e-03, PNorm = 68.4782, GNorm = 3.3267, lr_0 = 8.7691e-04
Validation prc-auc = 0.342311
Epoch 85
Validation prc-auc = 0.331255
Epoch 86
Validation prc-auc = 0.334936
Epoch 87
Validation prc-auc = 0.335591
Epoch 88
Validation prc-auc = 0.374555
Epoch 89
Loss = 2.3292e-03, PNorm = 68.5521, GNorm = 1.7709, lr_0 = 8.6941e-04
Validation prc-auc = 0.373894
Epoch 90
Validation prc-auc = 0.375441
Epoch 91
Validation prc-auc = 0.339491
Epoch 92
Validation prc-auc = 0.336218
Epoch 93
Validation prc-auc = 0.336766
Epoch 94
Loss = 8.5140e-04, PNorm = 68.6126, GNorm = 0.8489, lr_0 = 8.6198e-04
Validation prc-auc = 0.302661
Epoch 95
Validation prc-auc = 0.249875
Epoch 96
Validation prc-auc = 0.245395
Epoch 97
Validation prc-auc = 0.316017
Epoch 98
Validation prc-auc = 0.324420
Epoch 99
Loss = 1.6263e-03, PNorm = 68.7302, GNorm = 0.6045, lr_0 = 8.5461e-04
Validation prc-auc = 0.330619
Epoch 100
Validation prc-auc = 0.372311
Epoch 101
Validation prc-auc = 0.390166
Epoch 102
Validation prc-auc = 0.391375
Epoch 103
Validation prc-auc = 0.329125
Epoch 104
Loss = 1.0967e-03, PNorm = 68.8241, GNorm = 1.3100, lr_0 = 8.4730e-04
Validation prc-auc = 0.334925
Epoch 105
Validation prc-auc = 0.314510
Epoch 106
Validation prc-auc = 0.297844
Epoch 107
Validation prc-auc = 0.279753
Epoch 108
Validation prc-auc = 0.272016
Epoch 109
Loss = 8.0818e-04, PNorm = 68.8975, GNorm = 1.0226, lr_0 = 8.4006e-04
Validation prc-auc = 0.281264
Epoch 110
Validation prc-auc = 0.299431
Epoch 111
Validation prc-auc = 0.353697
Epoch 112
Validation prc-auc = 0.391440
Epoch 113
Validation prc-auc = 0.389814
Epoch 114
Loss = 2.1877e-04, PNorm = 68.9659, GNorm = 0.4841, lr_0 = 8.3288e-04
Validation prc-auc = 0.388033
Epoch 115
Validation prc-auc = 0.372961
Epoch 116
Validation prc-auc = 0.367474
Epoch 117
Validation prc-auc = 0.337719
Epoch 118
Validation prc-auc = 0.281744
Epoch 119
Loss = 6.9275e-05, PNorm = 69.0119, GNorm = 0.1527, lr_0 = 8.2576e-04
Validation prc-auc = 0.269365
Epoch 120
Validation prc-auc = 0.263497
Epoch 121
Validation prc-auc = 0.253907
Epoch 122
Validation prc-auc = 0.264533
Epoch 123
Validation prc-auc = 0.266992
Epoch 124
Loss = 1.0687e-04, PNorm = 69.0589, GNorm = 0.2515, lr_0 = 8.1870e-04
Validation prc-auc = 0.269078
Epoch 125
Validation prc-auc = 0.264465
Epoch 126
Validation prc-auc = 0.270322
Epoch 127
Validation prc-auc = 0.267545
Epoch 128
Validation prc-auc = 0.318990
Epoch 129
Loss = 4.0493e-04, PNorm = 69.1013, GNorm = 0.8532, lr_0 = 8.1170e-04
Validation prc-auc = 0.347872
Epoch 130
Validation prc-auc = 0.348556
Epoch 131
Validation prc-auc = 0.359926
Epoch 132
Validation prc-auc = 0.369613
Epoch 133
Validation prc-auc = 0.388040
Epoch 134
Loss = 5.1769e-04, PNorm = 69.1550, GNorm = 1.0234, lr_0 = 8.0476e-04
Validation prc-auc = 0.389035
Epoch 135
Validation prc-auc = 0.392855
Epoch 136
Validation prc-auc = 0.395627
Epoch 137
Validation prc-auc = 0.395504
Epoch 138
Validation prc-auc = 0.416787
Epoch 139
Loss = 4.4109e-04, PNorm = 69.2035, GNorm = 1.6315, lr_0 = 7.9788e-04
Validation prc-auc = 0.413844
Epoch 140
Validation prc-auc = 0.409017
Epoch 141
Validation prc-auc = 0.401789
Epoch 142
Validation prc-auc = 0.392406
Epoch 143
Validation prc-auc = 0.367336
Epoch 144
Loss = 1.3484e-04, PNorm = 69.2449, GNorm = 0.0915, lr_0 = 7.9106e-04
Validation prc-auc = 0.357047
Epoch 145
Validation prc-auc = 0.346643
Epoch 146
Validation prc-auc = 0.317600
Epoch 147
Validation prc-auc = 0.315663
Epoch 148
Validation prc-auc = 0.314100
Epoch 149
Loss = 6.3139e-04, PNorm = 69.3069, GNorm = 1.5617, lr_0 = 7.8430e-04
Validation prc-auc = 0.314914
Epoch 150
Validation prc-auc = 0.310705
Epoch 151
Validation prc-auc = 0.340091
Epoch 152
Validation prc-auc = 0.337817
Epoch 153
Validation prc-auc = 0.339487
Epoch 154
Loss = 1.1276e-04, PNorm = 69.3744, GNorm = 0.1496, lr_0 = 7.7759e-04
Validation prc-auc = 0.352865
Epoch 155
Validation prc-auc = 0.369160
Epoch 156
Validation prc-auc = 0.381653
Epoch 157
Validation prc-auc = 0.387021
Epoch 158
Validation prc-auc = 0.379301
Epoch 159
Loss = 6.3976e-05, PNorm = 69.4096, GNorm = 0.2044, lr_0 = 7.7095e-04
Validation prc-auc = 0.379787
Epoch 160
Validation prc-auc = 0.375174
Epoch 161
Validation prc-auc = 0.373323
Epoch 162
Validation prc-auc = 0.373810
Epoch 163
Validation prc-auc = 0.371398
Epoch 164
Loss = 5.7002e-04, PNorm = 69.4377, GNorm = 0.1285, lr_0 = 7.6436e-04
Validation prc-auc = 0.374678
Epoch 165
Validation prc-auc = 0.372698
Epoch 166
Validation prc-auc = 0.365994
Epoch 167
Validation prc-auc = 0.361394
Epoch 168
Validation prc-auc = 0.361195
Epoch 169
Loss = 1.4125e-04, PNorm = 69.4682, GNorm = 0.5517, lr_0 = 7.5782e-04
Validation prc-auc = 0.365770
Epoch 170
Validation prc-auc = 0.379301
Epoch 171
Validation prc-auc = 0.417649
Epoch 172
Validation prc-auc = 0.414224
Epoch 173
Validation prc-auc = 0.394289
Epoch 174
Loss = 3.8568e-04, PNorm = 69.4984, GNorm = 0.3053, lr_0 = 7.5134e-04
Validation prc-auc = 0.399167
Epoch 175
Validation prc-auc = 0.398265
Epoch 176
Validation prc-auc = 0.411495
Epoch 177
Validation prc-auc = 0.411909
Epoch 178
Validation prc-auc = 0.408899
Epoch 179
Loss = 4.0566e-05, PNorm = 69.5401, GNorm = 0.0529, lr_0 = 7.4492e-04
Validation prc-auc = 0.408050
Epoch 180
Validation prc-auc = 0.409981
Epoch 181
Validation prc-auc = 0.393973
Epoch 182
Validation prc-auc = 0.375408
Epoch 183
Validation prc-auc = 0.362118
Epoch 184
Loss = 1.7719e-04, PNorm = 69.5863, GNorm = 0.7909, lr_0 = 7.3855e-04
Validation prc-auc = 0.360895
Epoch 185
Validation prc-auc = 0.363121
Epoch 186
Validation prc-auc = 0.363654
Epoch 187
Validation prc-auc = 0.365699
Epoch 188
Validation prc-auc = 0.358267
Epoch 189
Loss = 4.7090e-04, PNorm = 69.6366, GNorm = 0.1782, lr_0 = 7.3224e-04
Validation prc-auc = 0.336213
Epoch 190
Validation prc-auc = 0.326434
Epoch 191
Validation prc-auc = 0.324734
Epoch 192
Validation prc-auc = 0.333091
Epoch 193
Validation prc-auc = 0.366383
Epoch 194
Loss = 8.0507e-04, PNorm = 69.6736, GNorm = 2.4869, lr_0 = 7.2598e-04
Validation prc-auc = 0.410155
Epoch 195
Validation prc-auc = 0.424088
Epoch 196
Validation prc-auc = 0.436279
Epoch 197
Validation prc-auc = 0.430949
Epoch 198
Validation prc-auc = 0.408204
Epoch 199
Loss = 1.1585e-03, PNorm = 69.7160, GNorm = 0.1810, lr_0 = 7.1977e-04
Validation prc-auc = 0.401419
Epoch 200
Validation prc-auc = 0.386261
Epoch 201
Validation prc-auc = 0.374797
Epoch 202
Validation prc-auc = 0.387626
Epoch 203
Validation prc-auc = 0.394416
Epoch 204
Loss = 9.3980e-05, PNorm = 69.7703, GNorm = 0.3482, lr_0 = 7.1362e-04
Validation prc-auc = 0.406916
Epoch 205
Validation prc-auc = 0.410458
Epoch 206
Validation prc-auc = 0.416348
Epoch 207
Validation prc-auc = 0.432070
Epoch 208
Validation prc-auc = 0.334920
Epoch 209
Loss = 9.6601e-05, PNorm = 69.8061, GNorm = 0.1032, lr_0 = 7.0752e-04
Validation prc-auc = 0.329954
Epoch 210
Validation prc-auc = 0.311482
Epoch 211
Validation prc-auc = 0.310543
Epoch 212
Validation prc-auc = 0.290060
Epoch 213
Validation prc-auc = 0.295516
Epoch 214
Loss = 1.5213e-04, PNorm = 69.8774, GNorm = 0.3489, lr_0 = 7.0147e-04
Validation prc-auc = 0.298555
Epoch 215
Validation prc-auc = 0.304008
Epoch 216
Validation prc-auc = 0.310696
Epoch 217
Validation prc-auc = 0.325095
Epoch 218
Validation prc-auc = 0.408370
Epoch 219
Loss = 3.0186e-04, PNorm = 69.9512, GNorm = 0.6978, lr_0 = 6.9548e-04
Validation prc-auc = 0.430633
Epoch 220
Validation prc-auc = 0.449081
Best score so far is 0.449081 at epoch 220 
Epoch 221
Validation prc-auc = 0.460867
Best score so far is 0.460867 at epoch 221 
Epoch 222
Validation prc-auc = 0.457351
Epoch 223
Validation prc-auc = 0.467187
Best score so far is 0.467187 at epoch 223 
Epoch 224
Loss = 3.3108e-04, PNorm = 70.0059, GNorm = 0.8509, lr_0 = 6.8953e-04
Validation prc-auc = 0.468005
Best score so far is 0.468005 at epoch 224 
Epoch 225
Validation prc-auc = 0.458086
Epoch 226
Validation prc-auc = 0.451501
Epoch 227
Validation prc-auc = 0.441768
Epoch 228
Validation prc-auc = 0.437076
Epoch 229
Loss = 1.7237e-04, PNorm = 70.0473, GNorm = 0.8927, lr_0 = 6.8364e-04
Validation prc-auc = 0.436906
Epoch 230
Validation prc-auc = 0.438392
Epoch 231
Validation prc-auc = 0.433169
Epoch 232
Validation prc-auc = 0.425299
Epoch 233
Validation prc-auc = 0.423497
Epoch 234
Loss = 9.2189e-05, PNorm = 70.0769, GNorm = 0.1030, lr_0 = 6.7779e-04
Validation prc-auc = 0.413232
Epoch 235
Validation prc-auc = 0.408658
Epoch 236
Validation prc-auc = 0.406171
Epoch 237
Validation prc-auc = 0.408866
Epoch 238
Validation prc-auc = 0.395204
Epoch 239
Loss = 2.2509e-05, PNorm = 70.0959, GNorm = 0.0212, lr_0 = 6.7200e-04
Validation prc-auc = 0.387922
Epoch 240
Validation prc-auc = 0.384222
Epoch 241
Validation prc-auc = 0.381569
Epoch 242
Validation prc-auc = 0.384365
Epoch 243
Validation prc-auc = 0.384662
Epoch 244
Loss = 5.4613e-05, PNorm = 70.1080, GNorm = 0.2640, lr_0 = 6.6625e-04
Validation prc-auc = 0.388644
Epoch 245
Validation prc-auc = 0.390560
Epoch 246
Validation prc-auc = 0.391366
Epoch 247
Validation prc-auc = 0.393684
Epoch 248
Validation prc-auc = 0.395704
Epoch 249
Loss = 1.4654e-05, PNorm = 70.1218, GNorm = 0.0066, lr_0 = 6.6056e-04
Validation prc-auc = 0.399464
Epoch 250
Validation prc-auc = 0.405503
Epoch 251
Validation prc-auc = 0.409175
Epoch 252
Validation prc-auc = 0.412114
Epoch 253
Validation prc-auc = 0.414009
Epoch 254
Loss = 1.5954e-05, PNorm = 70.1315, GNorm = 0.0500, lr_0 = 6.5491e-04
Validation prc-auc = 0.415763
Epoch 255
Validation prc-auc = 0.415797
Epoch 256
Validation prc-auc = 0.418969
Epoch 257
Validation prc-auc = 0.419837
Epoch 258
Validation prc-auc = 0.419985
Epoch 259
Loss = 1.2341e-05, PNorm = 70.1398, GNorm = 0.0122, lr_0 = 6.4931e-04
Validation prc-auc = 0.420096
Epoch 260
Validation prc-auc = 0.421053
Epoch 261
Validation prc-auc = 0.420440
Epoch 262
Validation prc-auc = 0.421477
Epoch 263
Validation prc-auc = 0.430812
Epoch 264
Loss = 1.8285e-05, PNorm = 70.1513, GNorm = 0.0601, lr_0 = 6.4376e-04
Validation prc-auc = 0.432354
Epoch 265
Validation prc-auc = 0.434291
Epoch 266
Validation prc-auc = 0.436154
Epoch 267
Validation prc-auc = 0.436795
Epoch 268
Validation prc-auc = 0.437231
Epoch 269
Loss = 1.9816e-05, PNorm = 70.1614, GNorm = 0.0112, lr_0 = 6.3826e-04
Validation prc-auc = 0.437187
Epoch 270
Validation prc-auc = 0.437187
Epoch 271
Validation prc-auc = 0.437348
Epoch 272
Validation prc-auc = 0.433191
Epoch 273
Validation prc-auc = 0.428566
Epoch 274
Loss = 9.1962e-05, PNorm = 70.1713, GNorm = 0.4512, lr_0 = 6.3280e-04
Validation prc-auc = 0.428477
Epoch 275
Validation prc-auc = 0.426098
Epoch 276
Validation prc-auc = 0.431968
Epoch 277
Validation prc-auc = 0.451176
Epoch 278
Validation prc-auc = 0.446197
Epoch 279
Loss = 1.1220e-04, PNorm = 70.1877, GNorm = 0.5469, lr_0 = 6.2739e-04
Validation prc-auc = 0.438902
Epoch 280
Validation prc-auc = 0.430857
Epoch 281
Validation prc-auc = 0.428636
Epoch 282
Validation prc-auc = 0.405002
Epoch 283
Validation prc-auc = 0.401524
Epoch 284
Loss = 4.9641e-05, PNorm = 70.2045, GNorm = 0.4034, lr_0 = 6.2203e-04
Validation prc-auc = 0.398400
Epoch 285
Validation prc-auc = 0.392255
Epoch 286
Validation prc-auc = 0.382959
Epoch 287
Validation prc-auc = 0.383922
Epoch 288
Validation prc-auc = 0.379333
Epoch 289
Loss = 1.1894e-05, PNorm = 70.2249, GNorm = 0.0226, lr_0 = 6.1671e-04
Validation prc-auc = 0.378670
Epoch 290
Validation prc-auc = 0.385280
Epoch 291
Validation prc-auc = 0.396884
Epoch 292
Validation prc-auc = 0.425340
Epoch 293
Validation prc-auc = 0.445104
Epoch 294
Loss = 2.3203e-04, PNorm = 70.2387, GNorm = 0.2637, lr_0 = 6.1144e-04
Validation prc-auc = 0.446114
Epoch 295
Validation prc-auc = 0.450081
Epoch 296
Validation prc-auc = 0.431271
Epoch 297
Validation prc-auc = 0.415473
Epoch 298
Validation prc-auc = 0.381761
Epoch 299
Loss = 1.7716e-04, PNorm = 70.2617, GNorm = 0.9598, lr_0 = 6.0621e-04
Validation prc-auc = 0.319667
Model 0 best validation prc-auc = 0.468005 on epoch 224
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.320944
Ensemble test prc-auc = 0.320944
Fold 6
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_6',
 'save_smiles_splits': False,
 'seed': 6,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.355800
Best score so far is 0.355800 at epoch 0 
Epoch 1
Validation prc-auc = 0.364786
Best score so far is 0.364786 at epoch 1 
Epoch 2
Validation prc-auc = 0.368895
Best score so far is 0.368895 at epoch 2 
Epoch 3
Validation prc-auc = 0.372443
Best score so far is 0.372443 at epoch 3 
Epoch 4
Loss = 9.0998e-03, PNorm = 66.7776, GNorm = 1.6401, lr_0 = 6.5000e-04
Validation prc-auc = 0.355106
Epoch 5
Validation prc-auc = 0.305744
Epoch 6
Validation prc-auc = 0.282794
Epoch 7
Validation prc-auc = 0.325314
Epoch 8
Validation prc-auc = 0.309936
Epoch 9
Loss = 5.1854e-03, PNorm = 66.9331, GNorm = 2.3574, lr_0 = 9.9743e-04
Validation prc-auc = 0.346693
Epoch 10
Validation prc-auc = 0.368555
Epoch 11
Validation prc-auc = 0.330956
Epoch 12
Validation prc-auc = 0.367789
Epoch 13
Validation prc-auc = 0.374106
Best score so far is 0.374106 at epoch 13 
Epoch 14
Loss = 5.8927e-03, PNorm = 67.1159, GNorm = 1.7233, lr_0 = 9.8890e-04
Validation prc-auc = 0.300707
Epoch 15
Validation prc-auc = 0.290812
Epoch 16
Validation prc-auc = 0.362035
Epoch 17
Validation prc-auc = 0.384304
Best score so far is 0.384304 at epoch 17 
Epoch 18
Validation prc-auc = 0.373892
Epoch 19
Loss = 3.4258e-03, PNorm = 67.2720, GNorm = 1.7179, lr_0 = 9.8045e-04
Validation prc-auc = 0.362579
Epoch 20
Validation prc-auc = 0.350400
Epoch 21
Validation prc-auc = 0.359881
Epoch 22
Validation prc-auc = 0.390322
Best score so far is 0.390322 at epoch 22 
Epoch 23
Validation prc-auc = 0.378444
Epoch 24
Loss = 3.1756e-03, PNorm = 67.3905, GNorm = 2.8666, lr_0 = 9.7207e-04
Validation prc-auc = 0.398887
Best score so far is 0.398887 at epoch 24 
Epoch 25
Validation prc-auc = 0.432485
Best score so far is 0.432485 at epoch 25 
Epoch 26
Validation prc-auc = 0.394816
Epoch 27
Validation prc-auc = 0.290061
Epoch 28
Validation prc-auc = 0.281224
Epoch 29
Loss = 1.9694e-03, PNorm = 67.4997, GNorm = 1.1841, lr_0 = 9.6376e-04
Validation prc-auc = 0.353574
Epoch 30
Validation prc-auc = 0.378084
Epoch 31
Validation prc-auc = 0.428599
Epoch 32
Validation prc-auc = 0.449729
Best score so far is 0.449729 at epoch 32 
Epoch 33
Validation prc-auc = 0.428620
Epoch 34
Loss = 3.4611e-03, PNorm = 67.6063, GNorm = 2.0648, lr_0 = 9.5552e-04
Validation prc-auc = 0.409751
Epoch 35
Validation prc-auc = 0.437745
Epoch 36
Validation prc-auc = 0.452587
Best score so far is 0.452587 at epoch 36 
Epoch 37
Validation prc-auc = 0.478906
Best score so far is 0.478906 at epoch 37 
Epoch 38
Validation prc-auc = 0.473624
Epoch 39
Loss = 3.2513e-03, PNorm = 67.7010, GNorm = 2.5861, lr_0 = 9.4735e-04
Validation prc-auc = 0.448244
Epoch 40
Validation prc-auc = 0.416542
Epoch 41
Validation prc-auc = 0.347347
Epoch 42
Validation prc-auc = 0.379201
Epoch 43
Validation prc-auc = 0.370938
Epoch 44
Loss = 2.0442e-03, PNorm = 67.7939, GNorm = 1.5055, lr_0 = 9.3925e-04
Validation prc-auc = 0.379274
Epoch 45
Validation prc-auc = 0.369811
Epoch 46
Validation prc-auc = 0.400407
Epoch 47
Validation prc-auc = 0.415420
Epoch 48
Validation prc-auc = 0.422153
Epoch 49
Loss = 1.0599e-03, PNorm = 67.8823, GNorm = 0.8620, lr_0 = 9.3122e-04
Validation prc-auc = 0.436306
Epoch 50
Validation prc-auc = 0.433668
Epoch 51
Validation prc-auc = 0.409294
Epoch 52
Validation prc-auc = 0.352844
Epoch 53
Validation prc-auc = 0.336028
Epoch 54
Loss = 1.3016e-03, PNorm = 67.9718, GNorm = 0.8717, lr_0 = 9.2326e-04
Validation prc-auc = 0.337885
Epoch 55
Validation prc-auc = 0.279329
Epoch 56
Validation prc-auc = 0.325785
Epoch 57
Validation prc-auc = 0.318157
Epoch 58
Validation prc-auc = 0.327381
Epoch 59
Loss = 6.2683e-04, PNorm = 68.0661, GNorm = 0.4234, lr_0 = 9.1537e-04
Validation prc-auc = 0.334906
Epoch 60
Validation prc-auc = 0.359585
Epoch 61
Validation prc-auc = 0.375666
Epoch 62
Validation prc-auc = 0.391128
Epoch 63
Validation prc-auc = 0.418045
Epoch 64
Loss = 3.5045e-03, PNorm = 68.1319, GNorm = 2.7543, lr_0 = 9.0754e-04
Validation prc-auc = 0.413666
Epoch 65
Validation prc-auc = 0.401934
Epoch 66
Validation prc-auc = 0.406011
Epoch 67
Validation prc-auc = 0.376758
Epoch 68
Validation prc-auc = 0.367771
Epoch 69
Loss = 5.1981e-04, PNorm = 68.1940, GNorm = 0.9866, lr_0 = 8.9978e-04
Validation prc-auc = 0.360435
Epoch 70
Validation prc-auc = 0.403785
Epoch 71
Validation prc-auc = 0.406111
Epoch 72
Validation prc-auc = 0.417785
Epoch 73
Validation prc-auc = 0.424832
Epoch 74
Loss = 1.3721e-04, PNorm = 68.2724, GNorm = 0.0760, lr_0 = 8.9209e-04
Validation prc-auc = 0.410399
Epoch 75
Validation prc-auc = 0.429754
Epoch 76
Validation prc-auc = 0.441131
Epoch 77
Validation prc-auc = 0.445679
Epoch 78
Validation prc-auc = 0.458055
Epoch 79
Loss = 7.9685e-04, PNorm = 68.3349, GNorm = 1.4424, lr_0 = 8.8447e-04
Validation prc-auc = 0.443993
Epoch 80
Validation prc-auc = 0.429156
Epoch 81
Validation prc-auc = 0.380610
Epoch 82
Validation prc-auc = 0.341366
Epoch 83
Validation prc-auc = 0.330342
Epoch 84
Loss = 2.8120e-04, PNorm = 68.3986, GNorm = 0.1250, lr_0 = 8.7691e-04
Validation prc-auc = 0.353762
Epoch 85
Validation prc-auc = 0.363376
Epoch 86
Validation prc-auc = 0.387590
Epoch 87
Validation prc-auc = 0.385691
Epoch 88
Validation prc-auc = 0.356917
Epoch 89
Loss = 9.1011e-04, PNorm = 68.4605, GNorm = 2.3211, lr_0 = 8.6941e-04
Validation prc-auc = 0.353607
Epoch 90
Validation prc-auc = 0.339579
Epoch 91
Validation prc-auc = 0.344054
Epoch 92
Validation prc-auc = 0.340320
Epoch 93
Validation prc-auc = 0.342780
Epoch 94
Loss = 5.7677e-04, PNorm = 68.5105, GNorm = 0.6106, lr_0 = 8.6198e-04
Validation prc-auc = 0.389983
Epoch 95
Validation prc-auc = 0.391870
Epoch 96
Validation prc-auc = 0.423446
Epoch 97
Validation prc-auc = 0.458054
Epoch 98
Validation prc-auc = 0.400299
Epoch 99
Loss = 9.3422e-04, PNorm = 68.5854, GNorm = 1.1652, lr_0 = 8.5461e-04
Validation prc-auc = 0.321712
Epoch 100
Validation prc-auc = 0.305819
Epoch 101
Validation prc-auc = 0.289112
Epoch 102
Validation prc-auc = 0.274587
Epoch 103
Validation prc-auc = 0.286582
Epoch 104
Loss = 1.0498e-04, PNorm = 68.6814, GNorm = 0.1256, lr_0 = 8.4730e-04
Validation prc-auc = 0.292483
Epoch 105
Validation prc-auc = 0.300536
Epoch 106
Validation prc-auc = 0.310465
Epoch 107
Validation prc-auc = 0.308391
Epoch 108
Validation prc-auc = 0.330584
Epoch 109
Loss = 1.8126e-04, PNorm = 68.7735, GNorm = 0.3958, lr_0 = 8.4006e-04
Validation prc-auc = 0.414951
Epoch 110
Validation prc-auc = 0.426476
Epoch 111
Validation prc-auc = 0.389752
Epoch 112
Validation prc-auc = 0.373098
Epoch 113
Validation prc-auc = 0.359362
Epoch 114
Loss = 5.0756e-04, PNorm = 68.8818, GNorm = 2.0039, lr_0 = 8.3288e-04
Validation prc-auc = 0.361928
Epoch 115
Validation prc-auc = 0.365575
Epoch 116
Validation prc-auc = 0.378458
Epoch 117
Validation prc-auc = 0.396061
Epoch 118
Validation prc-auc = 0.413844
Epoch 119
Loss = 5.6880e-04, PNorm = 68.9437, GNorm = 1.3148, lr_0 = 8.2576e-04
Validation prc-auc = 0.400062
Epoch 120
Validation prc-auc = 0.382945
Epoch 121
Validation prc-auc = 0.358154
Epoch 122
Validation prc-auc = 0.359476
Epoch 123
Validation prc-auc = 0.309863
Epoch 124
Loss = 1.4365e-03, PNorm = 69.0375, GNorm = 2.6162, lr_0 = 8.1870e-04
Validation prc-auc = 0.374557
Epoch 125
Validation prc-auc = 0.372135
Epoch 126
Validation prc-auc = 0.375510
Epoch 127
Validation prc-auc = 0.309417
Epoch 128
Validation prc-auc = 0.297014
Epoch 129
Loss = 4.3282e-04, PNorm = 69.1836, GNorm = 0.8704, lr_0 = 8.1170e-04
Validation prc-auc = 0.288592
Epoch 130
Validation prc-auc = 0.282774
Epoch 131
Validation prc-auc = 0.278460
Epoch 132
Validation prc-auc = 0.274757
Epoch 133
Validation prc-auc = 0.292790
Epoch 134
Loss = 1.3783e-04, PNorm = 69.2874, GNorm = 0.1856, lr_0 = 8.0476e-04
Validation prc-auc = 0.305177
Epoch 135
Validation prc-auc = 0.322157
Epoch 136
Validation prc-auc = 0.325953
Epoch 137
Validation prc-auc = 0.328472
Epoch 138
Validation prc-auc = 0.385455
Epoch 139
Loss = 1.5769e-03, PNorm = 69.3583, GNorm = 0.0784, lr_0 = 7.9788e-04
Validation prc-auc = 0.371029
Epoch 140
Validation prc-auc = 0.355474
Epoch 141
Validation prc-auc = 0.350672
Epoch 142
Validation prc-auc = 0.357207
Epoch 143
Validation prc-auc = 0.367299
Epoch 144
Loss = 8.8463e-05, PNorm = 69.4103, GNorm = 0.0734, lr_0 = 7.9106e-04
Validation prc-auc = 0.407311
Epoch 145
Validation prc-auc = 0.420608
Epoch 146
Validation prc-auc = 0.413139
Epoch 147
Validation prc-auc = 0.427205
Epoch 148
Validation prc-auc = 0.435242
Epoch 149
Loss = 6.4333e-04, PNorm = 69.4801, GNorm = 0.0624, lr_0 = 7.8430e-04
Validation prc-auc = 0.436443
Epoch 150
Validation prc-auc = 0.409710
Epoch 151
Validation prc-auc = 0.392123
Epoch 152
Validation prc-auc = 0.390284
Epoch 153
Validation prc-auc = 0.395106
Epoch 154
Loss = 1.9205e-03, PNorm = 69.5770, GNorm = 0.0820, lr_0 = 7.7759e-04
Validation prc-auc = 0.407318
Epoch 155
Validation prc-auc = 0.412413
Epoch 156
Validation prc-auc = 0.396345
Epoch 157
Validation prc-auc = 0.396424
Epoch 158
Validation prc-auc = 0.404531
Epoch 159
Loss = 1.0059e-03, PNorm = 69.6636, GNorm = 2.7170, lr_0 = 7.7095e-04
Validation prc-auc = 0.409496
Epoch 160
Validation prc-auc = 0.409692
Epoch 161
Validation prc-auc = 0.413245
Epoch 162
Validation prc-auc = 0.446716
Epoch 163
Validation prc-auc = 0.451341
Epoch 164
Loss = 1.3943e-04, PNorm = 69.7283, GNorm = 0.0702, lr_0 = 7.6436e-04
Validation prc-auc = 0.454197
Epoch 165
Validation prc-auc = 0.454414
Epoch 166
Validation prc-auc = 0.445890
Epoch 167
Validation prc-auc = 0.428365
Epoch 168
Validation prc-auc = 0.443412
Epoch 169
Loss = 3.4383e-05, PNorm = 69.7819, GNorm = 0.0389, lr_0 = 7.5782e-04
Validation prc-auc = 0.440139
Epoch 170
Validation prc-auc = 0.414770
Epoch 171
Validation prc-auc = 0.403003
Epoch 172
Validation prc-auc = 0.398639
Epoch 173
Validation prc-auc = 0.400194
Epoch 174
Loss = 9.6925e-05, PNorm = 69.8461, GNorm = 0.2543, lr_0 = 7.5134e-04
Validation prc-auc = 0.419598
Epoch 175
Validation prc-auc = 0.423935
Epoch 176
Validation prc-auc = 0.428437
Epoch 177
Validation prc-auc = 0.441661
Epoch 178
Validation prc-auc = 0.425540
Epoch 179
Loss = 2.9157e-04, PNorm = 69.9070, GNorm = 1.3483, lr_0 = 7.4492e-04
Validation prc-auc = 0.419041
Epoch 180
Validation prc-auc = 0.406471
Epoch 181
Validation prc-auc = 0.392709
Epoch 182
Validation prc-auc = 0.378101
Epoch 183
Validation prc-auc = 0.369542
Epoch 184
Loss = 2.0353e-04, PNorm = 69.9590, GNorm = 0.5416, lr_0 = 7.3855e-04
Validation prc-auc = 0.374432
Epoch 185
Validation prc-auc = 0.396176
Epoch 186
Validation prc-auc = 0.397365
Epoch 187
Validation prc-auc = 0.407591
Epoch 188
Validation prc-auc = 0.406449
Epoch 189
Loss = 2.8238e-04, PNorm = 69.9950, GNorm = 0.3568, lr_0 = 7.3224e-04
Validation prc-auc = 0.406900
Epoch 190
Validation prc-auc = 0.411044
Epoch 191
Validation prc-auc = 0.414040
Epoch 192
Validation prc-auc = 0.412540
Epoch 193
Validation prc-auc = 0.413264
Epoch 194
Loss = 4.0329e-05, PNorm = 70.0241, GNorm = 0.0945, lr_0 = 7.2598e-04
Validation prc-auc = 0.414019
Epoch 195
Validation prc-auc = 0.409581
Epoch 196
Validation prc-auc = 0.414233
Epoch 197
Validation prc-auc = 0.411656
Epoch 198
Validation prc-auc = 0.404484
Epoch 199
Loss = 9.8836e-05, PNorm = 70.0569, GNorm = 0.0261, lr_0 = 7.1977e-04
Validation prc-auc = 0.404484
Epoch 200
Validation prc-auc = 0.404166
Epoch 201
Validation prc-auc = 0.422629
Epoch 202
Validation prc-auc = 0.448557
Epoch 203
Validation prc-auc = 0.449818
Epoch 204
Loss = 1.0347e-04, PNorm = 70.0925, GNorm = 0.0172, lr_0 = 7.1362e-04
Validation prc-auc = 0.467994
Epoch 205
Validation prc-auc = 0.472941
Epoch 206
Validation prc-auc = 0.474067
Epoch 207
Validation prc-auc = 0.453899
Epoch 208
Validation prc-auc = 0.448903
Epoch 209
Loss = 6.7089e-05, PNorm = 70.1477, GNorm = 0.2929, lr_0 = 7.0752e-04
Validation prc-auc = 0.451475
Epoch 210
Validation prc-auc = 0.430976
Epoch 211
Validation prc-auc = 0.427066
Epoch 212
Validation prc-auc = 0.441744
Epoch 213
Validation prc-auc = 0.430084
Epoch 214
Loss = 4.1650e-05, PNorm = 70.1910, GNorm = 0.1850, lr_0 = 7.0147e-04
Validation prc-auc = 0.439548
Epoch 215
Validation prc-auc = 0.425495
Epoch 216
Validation prc-auc = 0.400983
Epoch 217
Validation prc-auc = 0.387659
Epoch 218
Validation prc-auc = 0.378249
Epoch 219
Loss = 1.8765e-04, PNorm = 70.2324, GNorm = 0.1312, lr_0 = 6.9548e-04
Validation prc-auc = 0.357873
Epoch 220
Validation prc-auc = 0.340423
Epoch 221
Validation prc-auc = 0.394895
Epoch 222
Validation prc-auc = 0.402467
Epoch 223
Validation prc-auc = 0.445541
Epoch 224
Loss = 2.2585e-05, PNorm = 70.2727, GNorm = 0.0396, lr_0 = 6.8953e-04
Validation prc-auc = 0.437756
Epoch 225
Validation prc-auc = 0.437474
Epoch 226
Validation prc-auc = 0.417299
Epoch 227
Validation prc-auc = 0.419966
Epoch 228
Validation prc-auc = 0.418348
Epoch 229
Loss = 2.9478e-05, PNorm = 70.3020, GNorm = 0.0589, lr_0 = 6.8364e-04
Validation prc-auc = 0.417207
Epoch 230
Validation prc-auc = 0.413501
Epoch 231
Validation prc-auc = 0.407942
Epoch 232
Validation prc-auc = 0.412645
Epoch 233
Validation prc-auc = 0.407457
Epoch 234
Loss = 2.5432e-04, PNorm = 70.3300, GNorm = 1.1725, lr_0 = 6.7779e-04
Validation prc-auc = 0.398707
Epoch 235
Validation prc-auc = 0.390667
Epoch 236
Validation prc-auc = 0.382668
Epoch 237
Validation prc-auc = 0.374061
Epoch 238
Validation prc-auc = 0.369716
Epoch 239
Loss = 3.1687e-04, PNorm = 70.3539, GNorm = 1.9894, lr_0 = 6.7200e-04
Validation prc-auc = 0.368025
Epoch 240
Validation prc-auc = 0.368844
Epoch 241
Validation prc-auc = 0.370570
Epoch 242
Validation prc-auc = 0.372189
Epoch 243
Validation prc-auc = 0.370689
Epoch 244
Loss = 6.9352e-05, PNorm = 70.3709, GNorm = 0.3098, lr_0 = 6.6625e-04
Validation prc-auc = 0.369256
Epoch 245
Validation prc-auc = 0.371298
Epoch 246
Validation prc-auc = 0.380224
Epoch 247
Validation prc-auc = 0.412828
Epoch 248
Validation prc-auc = 0.417916
Epoch 249
Loss = 6.6995e-05, PNorm = 70.3901, GNorm = 0.2664, lr_0 = 6.6056e-04
Validation prc-auc = 0.420105
Epoch 250
Validation prc-auc = 0.423110
Epoch 251
Validation prc-auc = 0.429334
Epoch 252
Validation prc-auc = 0.432675
Epoch 253
Validation prc-auc = 0.433985
Epoch 254
Loss = 2.3737e-05, PNorm = 70.4291, GNorm = 0.1366, lr_0 = 6.5491e-04
Validation prc-auc = 0.433482
Epoch 255
Validation prc-auc = 0.430624
Epoch 256
Validation prc-auc = 0.430679
Epoch 257
Validation prc-auc = 0.432309
Epoch 258
Validation prc-auc = 0.433347
Epoch 259
Loss = 9.0379e-05, PNorm = 70.4799, GNorm = 0.0057, lr_0 = 6.4931e-04
Validation prc-auc = 0.440359
Epoch 260
Validation prc-auc = 0.441005
Epoch 261
Validation prc-auc = 0.434975
Epoch 262
Validation prc-auc = 0.437307
Epoch 263
Validation prc-auc = 0.442321
Epoch 264
Loss = 1.4216e-05, PNorm = 70.5136, GNorm = 0.0094, lr_0 = 6.4376e-04
Validation prc-auc = 0.442634
Epoch 265
Validation prc-auc = 0.446464
Epoch 266
Validation prc-auc = 0.446629
Epoch 267
Validation prc-auc = 0.446311
Epoch 268
Validation prc-auc = 0.448710
Epoch 269
Loss = 3.5729e-05, PNorm = 70.5316, GNorm = 0.1203, lr_0 = 6.3826e-04
Validation prc-auc = 0.442129
Epoch 270
Validation prc-auc = 0.439183
Epoch 271
Validation prc-auc = 0.437608
Epoch 272
Validation prc-auc = 0.429365
Epoch 273
Validation prc-auc = 0.428384
Epoch 274
Loss = 1.7547e-04, PNorm = 70.5460, GNorm = 0.7340, lr_0 = 6.3280e-04
Validation prc-auc = 0.429502
Epoch 275
Validation prc-auc = 0.427849
Epoch 276
Validation prc-auc = 0.429170
Epoch 277
Validation prc-auc = 0.423718
Epoch 278
Validation prc-auc = 0.427955
Epoch 279
Loss = 1.2023e-04, PNorm = 70.5610, GNorm = 0.1367, lr_0 = 6.2739e-04
Validation prc-auc = 0.436868
Epoch 280
Validation prc-auc = 0.440156
Epoch 281
Validation prc-auc = 0.446835
Epoch 282
Validation prc-auc = 0.459460
Epoch 283
Validation prc-auc = 0.461455
Epoch 284
Loss = 8.9617e-06, PNorm = 70.5761, GNorm = 0.0056, lr_0 = 6.2203e-04
Validation prc-auc = 0.462918
Epoch 285
Validation prc-auc = 0.439003
Epoch 286
Validation prc-auc = 0.438195
Epoch 287
Validation prc-auc = 0.428632
Epoch 288
Validation prc-auc = 0.426179
Epoch 289
Loss = 1.4614e-05, PNorm = 70.5848, GNorm = 0.0386, lr_0 = 6.1671e-04
Validation prc-auc = 0.419945
Epoch 290
Validation prc-auc = 0.419434
Epoch 291
Validation prc-auc = 0.419069
Epoch 292
Validation prc-auc = 0.419937
Epoch 293
Validation prc-auc = 0.416330
Epoch 294
Loss = 3.3675e-04, PNorm = 70.5914, GNorm = 1.5854, lr_0 = 6.1144e-04
Validation prc-auc = 0.423765
Epoch 295
Validation prc-auc = 0.445989
Epoch 296
Validation prc-auc = 0.455860
Epoch 297
Validation prc-auc = 0.457278
Epoch 298
Validation prc-auc = 0.476722
Epoch 299
Loss = 2.5901e-05, PNorm = 70.6029, GNorm = 0.1776, lr_0 = 6.0621e-04
Validation prc-auc = 0.474040
Model 0 best validation prc-auc = 0.478906 on epoch 37
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.546844
Ensemble test prc-auc = 0.546844
Fold 7
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_7',
 'save_smiles_splits': False,
 'seed': 7,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.414860
Best score so far is 0.414860 at epoch 0 
Epoch 1
Validation prc-auc = 0.382848
Epoch 2
Validation prc-auc = 0.453505
Best score so far is 0.453505 at epoch 2 
Epoch 3
Validation prc-auc = 0.379933
Epoch 4
Loss = 1.0006e-02, PNorm = 66.7732, GNorm = 2.0154, lr_0 = 6.5000e-04
Validation prc-auc = 0.357140
Epoch 5
Validation prc-auc = 0.312515
Epoch 6
Validation prc-auc = 0.313902
Epoch 7
Validation prc-auc = 0.333606
Epoch 8
Validation prc-auc = 0.354846
Epoch 9
Loss = 5.8439e-03, PNorm = 66.9238, GNorm = 1.8541, lr_0 = 9.9743e-04
Validation prc-auc = 0.424446
Epoch 10
Validation prc-auc = 0.412457
Epoch 11
Validation prc-auc = 0.376759
Epoch 12
Validation prc-auc = 0.368162
Epoch 13
Validation prc-auc = 0.365667
Epoch 14
Loss = 2.8383e-03, PNorm = 67.1058, GNorm = 1.5700, lr_0 = 9.8890e-04
Validation prc-auc = 0.374615
Epoch 15
Validation prc-auc = 0.376039
Epoch 16
Validation prc-auc = 0.370712
Epoch 17
Validation prc-auc = 0.349031
Epoch 18
Validation prc-auc = 0.369394
Epoch 19
Loss = 2.2680e-03, PNorm = 67.2590, GNorm = 2.5697, lr_0 = 9.8045e-04
Validation prc-auc = 0.360184
Epoch 20
Validation prc-auc = 0.379019
Epoch 21
Validation prc-auc = 0.424005
Epoch 22
Validation prc-auc = 0.444655
Epoch 23
Validation prc-auc = 0.432092
Epoch 24
Loss = 1.4105e-03, PNorm = 67.3856, GNorm = 0.8107, lr_0 = 9.7207e-04
Validation prc-auc = 0.384657
Epoch 25
Validation prc-auc = 0.351080
Epoch 26
Validation prc-auc = 0.364134
Epoch 27
Validation prc-auc = 0.335015
Epoch 28
Validation prc-auc = 0.420262
Epoch 29
Loss = 2.0004e-03, PNorm = 67.5024, GNorm = 1.1767, lr_0 = 9.6376e-04
Validation prc-auc = 0.391942
Epoch 30
Validation prc-auc = 0.359201
Epoch 31
Validation prc-auc = 0.314548
Epoch 32
Validation prc-auc = 0.312177
Epoch 33
Validation prc-auc = 0.334231
Epoch 34
Loss = 2.2812e-03, PNorm = 67.6010, GNorm = 2.1053, lr_0 = 9.5552e-04
Validation prc-auc = 0.376307
Epoch 35
Validation prc-auc = 0.364271
Epoch 36
Validation prc-auc = 0.315706
Epoch 37
Validation prc-auc = 0.305076
Epoch 38
Validation prc-auc = 0.321940
Epoch 39
Loss = 1.5758e-03, PNorm = 67.6937, GNorm = 0.7148, lr_0 = 9.4735e-04
Validation prc-auc = 0.378503
Epoch 40
Validation prc-auc = 0.395930
Epoch 41
Validation prc-auc = 0.408675
Epoch 42
Validation prc-auc = 0.401462
Epoch 43
Validation prc-auc = 0.382829
Epoch 44
Loss = 5.6016e-03, PNorm = 67.7757, GNorm = 4.2615, lr_0 = 9.3925e-04
Validation prc-auc = 0.390633
Epoch 45
Validation prc-auc = 0.389427
Epoch 46
Validation prc-auc = 0.396148
Epoch 47
Validation prc-auc = 0.382881
Epoch 48
Validation prc-auc = 0.403256
Epoch 49
Loss = 1.9330e-03, PNorm = 67.8432, GNorm = 1.0849, lr_0 = 9.3122e-04
Validation prc-auc = 0.459248
Best score so far is 0.459248 at epoch 49 
Epoch 50
Validation prc-auc = 0.478643
Best score so far is 0.478643 at epoch 50 
Epoch 51
Validation prc-auc = 0.479660
Best score so far is 0.479660 at epoch 51 
Epoch 52
Validation prc-auc = 0.467400
Epoch 53
Validation prc-auc = 0.382735
Epoch 54
Loss = 3.1710e-03, PNorm = 67.9320, GNorm = 1.7537, lr_0 = 9.2326e-04
Validation prc-auc = 0.334210
Epoch 55
Validation prc-auc = 0.288779
Epoch 56
Validation prc-auc = 0.364252
Epoch 57
Validation prc-auc = 0.381598
Epoch 58
Validation prc-auc = 0.412015
Epoch 59
Loss = 3.4147e-03, PNorm = 68.0328, GNorm = 1.7556, lr_0 = 9.1537e-04
Validation prc-auc = 0.446814
Epoch 60
Validation prc-auc = 0.454618
Epoch 61
Validation prc-auc = 0.397192
Epoch 62
Validation prc-auc = 0.322482
Epoch 63
Validation prc-auc = 0.254946
Epoch 64
Loss = 1.0978e-03, PNorm = 68.1481, GNorm = 1.6560, lr_0 = 9.0754e-04
Validation prc-auc = 0.288363
Epoch 65
Validation prc-auc = 0.313801
Epoch 66
Validation prc-auc = 0.312450
Epoch 67
Validation prc-auc = 0.299760
Epoch 68
Validation prc-auc = 0.298401
Epoch 69
Loss = 2.5254e-03, PNorm = 68.2577, GNorm = 2.3675, lr_0 = 8.9978e-04
Validation prc-auc = 0.372920
Epoch 70
Validation prc-auc = 0.392283
Epoch 71
Validation prc-auc = 0.388606
Epoch 72
Validation prc-auc = 0.404851
Epoch 73
Validation prc-auc = 0.414310
Epoch 74
Loss = 1.9321e-03, PNorm = 68.3485, GNorm = 0.3831, lr_0 = 8.9209e-04
Validation prc-auc = 0.409049
Epoch 75
Validation prc-auc = 0.436576
Epoch 76
Validation prc-auc = 0.429897
Epoch 77
Validation prc-auc = 0.428539
Epoch 78
Validation prc-auc = 0.428000
Epoch 79
Loss = 5.0742e-04, PNorm = 68.4341, GNorm = 0.4376, lr_0 = 8.8447e-04
Validation prc-auc = 0.421585
Epoch 80
Validation prc-auc = 0.408878
Epoch 81
Validation prc-auc = 0.385050
Epoch 82
Validation prc-auc = 0.376704
Epoch 83
Validation prc-auc = 0.389267
Epoch 84
Loss = 1.8539e-03, PNorm = 68.5032, GNorm = 1.9023, lr_0 = 8.7691e-04
Validation prc-auc = 0.392990
Epoch 85
Validation prc-auc = 0.408641
Epoch 86
Validation prc-auc = 0.391651
Epoch 87
Validation prc-auc = 0.417186
Epoch 88
Validation prc-auc = 0.411591
Epoch 89
Loss = 3.3716e-04, PNorm = 68.5884, GNorm = 0.4034, lr_0 = 8.6941e-04
Validation prc-auc = 0.413014
Epoch 90
Validation prc-auc = 0.411121
Epoch 91
Validation prc-auc = 0.412502
Epoch 92
Validation prc-auc = 0.399366
Epoch 93
Validation prc-auc = 0.407105
Epoch 94
Loss = 2.0736e-04, PNorm = 68.6579, GNorm = 0.1746, lr_0 = 8.6198e-04
Validation prc-auc = 0.408932
Epoch 95
Validation prc-auc = 0.420335
Epoch 96
Validation prc-auc = 0.427113
Epoch 97
Validation prc-auc = 0.426798
Epoch 98
Validation prc-auc = 0.393075
Epoch 99
Loss = 1.9099e-04, PNorm = 68.7161, GNorm = 0.4544, lr_0 = 8.5461e-04
Validation prc-auc = 0.387516
Epoch 100
Validation prc-auc = 0.377347
Epoch 101
Validation prc-auc = 0.377851
Epoch 102
Validation prc-auc = 0.401522
Epoch 103
Validation prc-auc = 0.428209
Epoch 104
Loss = 1.9789e-04, PNorm = 68.7607, GNorm = 0.3068, lr_0 = 8.4730e-04
Validation prc-auc = 0.426163
Epoch 105
Validation prc-auc = 0.437330
Epoch 106
Validation prc-auc = 0.451401
Epoch 107
Validation prc-auc = 0.451109
Epoch 108
Validation prc-auc = 0.430349
Epoch 109
Loss = 8.6569e-04, PNorm = 68.8037, GNorm = 1.2091, lr_0 = 8.4006e-04
Validation prc-auc = 0.415258
Epoch 110
Validation prc-auc = 0.396979
Epoch 111
Validation prc-auc = 0.392357
Epoch 112
Validation prc-auc = 0.400850
Epoch 113
Validation prc-auc = 0.407716
Epoch 114
Loss = 2.0847e-03, PNorm = 68.8595, GNorm = 1.3911, lr_0 = 8.3288e-04
Validation prc-auc = 0.400536
Epoch 115
Validation prc-auc = 0.394679
Epoch 116
Validation prc-auc = 0.391827
Epoch 117
Validation prc-auc = 0.392673
Epoch 118
Validation prc-auc = 0.398246
Epoch 119
Loss = 4.0415e-04, PNorm = 68.9097, GNorm = 0.2862, lr_0 = 8.2576e-04
Validation prc-auc = 0.390167
Epoch 120
Validation prc-auc = 0.380954
Epoch 121
Validation prc-auc = 0.407986
Epoch 122
Validation prc-auc = 0.417659
Epoch 123
Validation prc-auc = 0.431733
Epoch 124
Loss = 1.7291e-04, PNorm = 68.9491, GNorm = 0.0401, lr_0 = 8.1870e-04
Validation prc-auc = 0.438934
Epoch 125
Validation prc-auc = 0.434926
Epoch 126
Validation prc-auc = 0.425630
Epoch 127
Validation prc-auc = 0.422897
Epoch 128
Validation prc-auc = 0.403186
Epoch 129
Loss = 1.8534e-04, PNorm = 69.0140, GNorm = 0.5129, lr_0 = 8.1170e-04
Validation prc-auc = 0.402134
Epoch 130
Validation prc-auc = 0.392616
Epoch 131
Validation prc-auc = 0.390600
Epoch 132
Validation prc-auc = 0.408971
Epoch 133
Validation prc-auc = 0.425929
Epoch 134
Loss = 1.4598e-04, PNorm = 69.0869, GNorm = 0.2017, lr_0 = 8.0476e-04
Validation prc-auc = 0.435031
Epoch 135
Validation prc-auc = 0.442469
Epoch 136
Validation prc-auc = 0.472272
Epoch 137
Validation prc-auc = 0.474034
Epoch 138
Validation prc-auc = 0.472990
Epoch 139
Loss = 1.1002e-04, PNorm = 69.1376, GNorm = 0.2020, lr_0 = 7.9788e-04
Validation prc-auc = 0.452171
Epoch 140
Validation prc-auc = 0.447823
Epoch 141
Validation prc-auc = 0.446772
Epoch 142
Validation prc-auc = 0.422147
Epoch 143
Validation prc-auc = 0.404913
Epoch 144
Loss = 8.8734e-05, PNorm = 69.1851, GNorm = 0.1426, lr_0 = 7.9106e-04
Validation prc-auc = 0.405451
Epoch 145
Validation prc-auc = 0.402284
Epoch 146
Validation prc-auc = 0.403327
Epoch 147
Validation prc-auc = 0.409315
Epoch 148
Validation prc-auc = 0.422432
Epoch 149
Loss = 1.8122e-04, PNorm = 69.2284, GNorm = 0.0552, lr_0 = 7.8430e-04
Validation prc-auc = 0.436111
Epoch 150
Validation prc-auc = 0.434870
Epoch 151
Validation prc-auc = 0.432766
Epoch 152
Validation prc-auc = 0.436342
Epoch 153
Validation prc-auc = 0.436107
Epoch 154
Loss = 1.9674e-04, PNorm = 69.2776, GNorm = 1.0832, lr_0 = 7.7759e-04
Validation prc-auc = 0.423337
Epoch 155
Validation prc-auc = 0.417073
Epoch 156
Validation prc-auc = 0.442698
Epoch 157
Validation prc-auc = 0.447974
Epoch 158
Validation prc-auc = 0.454994
Epoch 159
Loss = 1.1903e-03, PNorm = 69.3107, GNorm = 2.5939, lr_0 = 7.7095e-04
Validation prc-auc = 0.478852
Epoch 160
Validation prc-auc = 0.479533
Epoch 161
Validation prc-auc = 0.474537
Epoch 162
Validation prc-auc = 0.468347
Epoch 163
Validation prc-auc = 0.454022
Epoch 164
Loss = 2.3795e-04, PNorm = 69.3778, GNorm = 0.6937, lr_0 = 7.6436e-04
Validation prc-auc = 0.436363
Epoch 165
Validation prc-auc = 0.400710
Epoch 166
Validation prc-auc = 0.324041
Epoch 167
Validation prc-auc = 0.311214
Epoch 168
Validation prc-auc = 0.297383
Epoch 169
Loss = 8.6141e-05, PNorm = 69.4636, GNorm = 0.2886, lr_0 = 7.5782e-04
Validation prc-auc = 0.297365
Epoch 170
Validation prc-auc = 0.309506
Epoch 171
Validation prc-auc = 0.316741
Epoch 172
Validation prc-auc = 0.323470
Epoch 173
Validation prc-auc = 0.383284
Epoch 174
Loss = 6.7230e-05, PNorm = 69.5190, GNorm = 0.2367, lr_0 = 7.5134e-04
Validation prc-auc = 0.393522
Epoch 175
Validation prc-auc = 0.403788
Epoch 176
Validation prc-auc = 0.410044
Epoch 177
Validation prc-auc = 0.407983
Epoch 178
Validation prc-auc = 0.404987
Epoch 179
Loss = 1.5942e-04, PNorm = 69.5661, GNorm = 0.3404, lr_0 = 7.4492e-04
Validation prc-auc = 0.403499
Epoch 180
Validation prc-auc = 0.420099
Epoch 181
Validation prc-auc = 0.417281
Epoch 182
Validation prc-auc = 0.413550
Epoch 183
Validation prc-auc = 0.418381
Epoch 184
Loss = 6.9828e-04, PNorm = 69.6189, GNorm = 2.0607, lr_0 = 7.3855e-04
Validation prc-auc = 0.424500
Epoch 185
Validation prc-auc = 0.424532
Epoch 186
Validation prc-auc = 0.422191
Epoch 187
Validation prc-auc = 0.426987
Epoch 188
Validation prc-auc = 0.427070
Epoch 189
Loss = 1.7408e-04, PNorm = 69.6805, GNorm = 0.0382, lr_0 = 7.3224e-04
Validation prc-auc = 0.422472
Epoch 190
Validation prc-auc = 0.388514
Epoch 191
Validation prc-auc = 0.384858
Epoch 192
Validation prc-auc = 0.395403
Epoch 193
Validation prc-auc = 0.400425
Epoch 194
Loss = 2.9520e-04, PNorm = 69.7494, GNorm = 1.1852, lr_0 = 7.2598e-04
Validation prc-auc = 0.380781
Epoch 195
Validation prc-auc = 0.312442
Epoch 196
Validation prc-auc = 0.295741
Epoch 197
Validation prc-auc = 0.386694
Epoch 198
Validation prc-auc = 0.389236
Epoch 199
Loss = 1.6125e-03, PNorm = 69.8050, GNorm = 1.5911, lr_0 = 7.1977e-04
Validation prc-auc = 0.415764
Epoch 200
Validation prc-auc = 0.434472
Epoch 201
Validation prc-auc = 0.452774
Epoch 202
Validation prc-auc = 0.461976
Epoch 203
Validation prc-auc = 0.438849
Epoch 204
Loss = 4.8811e-04, PNorm = 69.8527, GNorm = 1.8133, lr_0 = 7.1362e-04
Validation prc-auc = 0.405631
Epoch 205
Validation prc-auc = 0.371733
Epoch 206
Validation prc-auc = 0.314873
Epoch 207
Validation prc-auc = 0.309420
Epoch 208
Validation prc-auc = 0.305425
Epoch 209
Loss = 3.4992e-04, PNorm = 69.9104, GNorm = 0.0560, lr_0 = 7.0752e-04
Validation prc-auc = 0.304041
Epoch 210
Validation prc-auc = 0.301617
Epoch 211
Validation prc-auc = 0.301519
Epoch 212
Validation prc-auc = 0.290876
Epoch 213
Validation prc-auc = 0.289044
Epoch 214
Loss = 5.7750e-04, PNorm = 69.9883, GNorm = 0.9087, lr_0 = 7.0147e-04
Validation prc-auc = 0.269780
Epoch 215
Validation prc-auc = 0.255683
Epoch 216
Validation prc-auc = 0.256869
Epoch 217
Validation prc-auc = 0.253901
Epoch 218
Validation prc-auc = 0.256872
Epoch 219
Loss = 6.7676e-05, PNorm = 70.0610, GNorm = 0.0262, lr_0 = 6.9548e-04
Validation prc-auc = 0.261577
Epoch 220
Validation prc-auc = 0.265014
Epoch 221
Validation prc-auc = 0.266731
Epoch 222
Validation prc-auc = 0.267331
Epoch 223
Validation prc-auc = 0.272976
Epoch 224
Loss = 7.1778e-05, PNorm = 70.1037, GNorm = 0.3268, lr_0 = 6.8953e-04
Validation prc-auc = 0.273128
Epoch 225
Validation prc-auc = 0.279146
Epoch 226
Validation prc-auc = 0.279441
Epoch 227
Validation prc-auc = 0.328576
Epoch 228
Validation prc-auc = 0.356110
Epoch 229
Loss = 6.0391e-04, PNorm = 70.1237, GNorm = 1.5994, lr_0 = 6.8364e-04
Validation prc-auc = 0.378218
Epoch 230
Validation prc-auc = 0.385604
Epoch 231
Validation prc-auc = 0.405748
Epoch 232
Validation prc-auc = 0.420076
Epoch 233
Validation prc-auc = 0.418090
Epoch 234
Loss = 1.8790e-04, PNorm = 70.1529, GNorm = 0.6255, lr_0 = 6.7779e-04
Validation prc-auc = 0.420165
Epoch 235
Validation prc-auc = 0.420272
Epoch 236
Validation prc-auc = 0.450939
Epoch 237
Validation prc-auc = 0.450178
Epoch 238
Validation prc-auc = 0.443270
Epoch 239
Loss = 2.7572e-05, PNorm = 70.1855, GNorm = 0.0835, lr_0 = 6.7200e-04
Validation prc-auc = 0.440229
Epoch 240
Validation prc-auc = 0.432981
Epoch 241
Validation prc-auc = 0.432123
Epoch 242
Validation prc-auc = 0.431430
Epoch 243
Validation prc-auc = 0.397143
Epoch 244
Loss = 5.2860e-05, PNorm = 70.2143, GNorm = 0.0759, lr_0 = 6.6625e-04
Validation prc-auc = 0.395326
Epoch 245
Validation prc-auc = 0.393493
Epoch 246
Validation prc-auc = 0.390860
Epoch 247
Validation prc-auc = 0.389252
Epoch 248
Validation prc-auc = 0.388725
Epoch 249
Loss = 3.0520e-04, PNorm = 70.2338, GNorm = 0.1237, lr_0 = 6.6056e-04
Validation prc-auc = 0.381649
Epoch 250
Validation prc-auc = 0.371100
Epoch 251
Validation prc-auc = 0.361900
Epoch 252
Validation prc-auc = 0.362380
Epoch 253
Validation prc-auc = 0.358180
Epoch 254
Loss = 5.1247e-05, PNorm = 70.2535, GNorm = 0.1183, lr_0 = 6.5491e-04
Validation prc-auc = 0.356975
Epoch 255
Validation prc-auc = 0.364002
Epoch 256
Validation prc-auc = 0.364575
Epoch 257
Validation prc-auc = 0.368187
Epoch 258
Validation prc-auc = 0.372605
Epoch 259
Loss = 2.6020e-04, PNorm = 70.2715, GNorm = 1.3329, lr_0 = 6.4931e-04
Validation prc-auc = 0.379369
Epoch 260
Validation prc-auc = 0.384612
Epoch 261
Validation prc-auc = 0.385693
Epoch 262
Validation prc-auc = 0.384365
Epoch 263
Validation prc-auc = 0.384991
Epoch 264
Loss = 2.0422e-05, PNorm = 70.2897, GNorm = 0.0487, lr_0 = 6.4376e-04
Validation prc-auc = 0.391384
Epoch 265
Validation prc-auc = 0.395244
Epoch 266
Validation prc-auc = 0.394559
Epoch 267
Validation prc-auc = 0.394389
Epoch 268
Validation prc-auc = 0.394005
Epoch 269
Loss = 1.7098e-05, PNorm = 70.3047, GNorm = 0.0138, lr_0 = 6.3826e-04
Validation prc-auc = 0.394690
Epoch 270
Validation prc-auc = 0.393187
Epoch 271
Validation prc-auc = 0.393926
Epoch 272
Validation prc-auc = 0.397014
Epoch 273
Validation prc-auc = 0.397715
Epoch 274
Loss = 7.8216e-06, PNorm = 70.3114, GNorm = 0.0235, lr_0 = 6.3280e-04
Validation prc-auc = 0.397715
Epoch 275
Validation prc-auc = 0.399637
Epoch 276
Validation prc-auc = 0.401182
Epoch 277
Validation prc-auc = 0.401439
Epoch 278
Validation prc-auc = 0.404580
Epoch 279
Loss = 1.2950e-05, PNorm = 70.3137, GNorm = 0.0169, lr_0 = 6.2739e-04
Validation prc-auc = 0.401503
Epoch 280
Validation prc-auc = 0.398591
Epoch 281
Validation prc-auc = 0.398591
Epoch 282
Validation prc-auc = 0.397376
Epoch 283
Validation prc-auc = 0.399100
Epoch 284
Loss = 5.9025e-04, PNorm = 70.3238, GNorm = 0.4349, lr_0 = 6.2203e-04
Validation prc-auc = 0.404626
Epoch 285
Validation prc-auc = 0.405661
Epoch 286
Validation prc-auc = 0.405363
Epoch 287
Validation prc-auc = 0.406587
Epoch 288
Validation prc-auc = 0.411773
Epoch 289
Loss = 1.2219e-04, PNorm = 70.3554, GNorm = 0.0916, lr_0 = 6.1671e-04
Validation prc-auc = 0.409203
Epoch 290
Validation prc-auc = 0.410100
Epoch 291
Validation prc-auc = 0.408550
Epoch 292
Validation prc-auc = 0.413945
Epoch 293
Validation prc-auc = 0.403927
Epoch 294
Loss = 4.2180e-06, PNorm = 70.3980, GNorm = 0.0016, lr_0 = 6.1144e-04
Validation prc-auc = 0.408627
Epoch 295
Validation prc-auc = 0.406544
Epoch 296
Validation prc-auc = 0.404733
Epoch 297
Validation prc-auc = 0.404034
Epoch 298
Validation prc-auc = 0.405105
Epoch 299
Loss = 2.3090e-05, PNorm = 70.4239, GNorm = 0.0813, lr_0 = 6.0621e-04
Validation prc-auc = 0.405048
Model 0 best validation prc-auc = 0.479660 on epoch 51
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.444717
Ensemble test prc-auc = 0.444717
Fold 8
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_8',
 'save_smiles_splits': False,
 'seed': 8,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.391165
Best score so far is 0.391165 at epoch 0 
Epoch 1
Validation prc-auc = 0.366213
Epoch 2
Validation prc-auc = 0.382161
Epoch 3
Validation prc-auc = 0.377428
Epoch 4
Loss = 8.2612e-03, PNorm = 66.7761, GNorm = 1.6500, lr_0 = 6.5000e-04
Validation prc-auc = 0.354203
Epoch 5
Validation prc-auc = 0.306931
Epoch 6
Validation prc-auc = 0.306839
Epoch 7
Validation prc-auc = 0.323344
Epoch 8
Validation prc-auc = 0.372517
Epoch 9
Loss = 5.9090e-03, PNorm = 66.9370, GNorm = 1.7124, lr_0 = 9.9743e-04
Validation prc-auc = 0.352260
Epoch 10
Validation prc-auc = 0.258614
Epoch 11
Validation prc-auc = 0.281627
Epoch 12
Validation prc-auc = 0.299328
Epoch 13
Validation prc-auc = 0.330307
Epoch 14
Loss = 5.5651e-03, PNorm = 67.1254, GNorm = 3.7188, lr_0 = 9.8890e-04
Validation prc-auc = 0.356292
Epoch 15
Validation prc-auc = 0.330847
Epoch 16
Validation prc-auc = 0.411312
Best score so far is 0.411312 at epoch 16 
Epoch 17
Validation prc-auc = 0.389746
Epoch 18
Validation prc-auc = 0.374215
Epoch 19
Loss = 2.2223e-03, PNorm = 67.2799, GNorm = 1.6381, lr_0 = 9.8045e-04
Validation prc-auc = 0.286428
Epoch 20
Validation prc-auc = 0.277927
Epoch 21
Validation prc-auc = 0.335255
Epoch 22
Validation prc-auc = 0.305141
Epoch 23
Validation prc-auc = 0.285477
Epoch 24
Loss = 3.6721e-03, PNorm = 67.4142, GNorm = 2.9347, lr_0 = 9.7207e-04
Validation prc-auc = 0.305869
Epoch 25
Validation prc-auc = 0.300049
Epoch 26
Validation prc-auc = 0.258603
Epoch 27
Validation prc-auc = 0.242953
Epoch 28
Validation prc-auc = 0.221262
Epoch 29
Loss = 3.7180e-03, PNorm = 67.5268, GNorm = 2.0256, lr_0 = 9.6376e-04
Validation prc-auc = 0.210165
Epoch 30
Validation prc-auc = 0.243526
Epoch 31
Validation prc-auc = 0.327874
Epoch 32
Validation prc-auc = 0.351676
Epoch 33
Validation prc-auc = 0.363948
Epoch 34
Loss = 2.8154e-03, PNorm = 67.6444, GNorm = 1.3225, lr_0 = 9.5552e-04
Validation prc-auc = 0.397059
Epoch 35
Validation prc-auc = 0.360000
Epoch 36
Validation prc-auc = 0.330987
Epoch 37
Validation prc-auc = 0.326115
Epoch 38
Validation prc-auc = 0.356481
Epoch 39
Loss = 1.9747e-03, PNorm = 67.7435, GNorm = 1.3486, lr_0 = 9.4735e-04
Validation prc-auc = 0.374854
Epoch 40
Validation prc-auc = 0.381182
Epoch 41
Validation prc-auc = 0.322425
Epoch 42
Validation prc-auc = 0.251454
Epoch 43
Validation prc-auc = 0.271793
Epoch 44
Loss = 1.7444e-03, PNorm = 67.8307, GNorm = 1.0408, lr_0 = 9.3925e-04
Validation prc-auc = 0.279254
Epoch 45
Validation prc-auc = 0.364756
Epoch 46
Validation prc-auc = 0.400218
Epoch 47
Validation prc-auc = 0.428607
Best score so far is 0.428607 at epoch 47 
Epoch 48
Validation prc-auc = 0.415838
Epoch 49
Loss = 1.5375e-03, PNorm = 67.9352, GNorm = 0.6993, lr_0 = 9.3122e-04
Validation prc-auc = 0.349245
Epoch 50
Validation prc-auc = 0.319229
Epoch 51
Validation prc-auc = 0.300114
Epoch 52
Validation prc-auc = 0.298078
Epoch 53
Validation prc-auc = 0.312989
Epoch 54
Loss = 8.1786e-04, PNorm = 68.0215, GNorm = 1.4867, lr_0 = 9.2326e-04
Validation prc-auc = 0.349431
Epoch 55
Validation prc-auc = 0.346538
Epoch 56
Validation prc-auc = 0.355318
Epoch 57
Validation prc-auc = 0.352059
Epoch 58
Validation prc-auc = 0.343053
Epoch 59
Loss = 1.3450e-03, PNorm = 68.1012, GNorm = 1.4551, lr_0 = 9.1537e-04
Validation prc-auc = 0.362013
Epoch 60
Validation prc-auc = 0.370949
Epoch 61
Validation prc-auc = 0.371934
Epoch 62
Validation prc-auc = 0.333526
Epoch 63
Validation prc-auc = 0.328801
Epoch 64
Loss = 5.9942e-04, PNorm = 68.1752, GNorm = 0.6260, lr_0 = 9.0754e-04
Validation prc-auc = 0.340841
Epoch 65
Validation prc-auc = 0.387334
Epoch 66
Validation prc-auc = 0.417114
Epoch 67
Validation prc-auc = 0.400366
Epoch 68
Validation prc-auc = 0.376866
Epoch 69
Loss = 3.2173e-04, PNorm = 68.2447, GNorm = 0.3455, lr_0 = 8.9978e-04
Validation prc-auc = 0.368451
Epoch 70
Validation prc-auc = 0.360362
Epoch 71
Validation prc-auc = 0.260386
Epoch 72
Validation prc-auc = 0.268956
Epoch 73
Validation prc-auc = 0.278916
Epoch 74
Loss = 2.0611e-04, PNorm = 68.3076, GNorm = 0.3005, lr_0 = 8.9209e-04
Validation prc-auc = 0.323542
Epoch 75
Validation prc-auc = 0.353008
Epoch 76
Validation prc-auc = 0.360554
Epoch 77
Validation prc-auc = 0.366136
Epoch 78
Validation prc-auc = 0.353986
Epoch 79
Loss = 1.5043e-03, PNorm = 68.3905, GNorm = 0.3536, lr_0 = 8.8447e-04
Validation prc-auc = 0.368231
Epoch 80
Validation prc-auc = 0.420776
Epoch 81
Validation prc-auc = 0.420879
Epoch 82
Validation prc-auc = 0.401106
Epoch 83
Validation prc-auc = 0.395889
Epoch 84
Loss = 5.0872e-04, PNorm = 68.4638, GNorm = 1.1105, lr_0 = 8.7691e-04
Validation prc-auc = 0.385041
Epoch 85
Validation prc-auc = 0.369854
Epoch 86
Validation prc-auc = 0.359188
Epoch 87
Validation prc-auc = 0.349895
Epoch 88
Validation prc-auc = 0.347854
Epoch 89
Loss = 5.7887e-04, PNorm = 68.5466, GNorm = 0.1689, lr_0 = 8.6941e-04
Validation prc-auc = 0.337406
Epoch 90
Validation prc-auc = 0.347081
Epoch 91
Validation prc-auc = 0.319561
Epoch 92
Validation prc-auc = 0.257483
Epoch 93
Validation prc-auc = 0.254575
Epoch 94
Loss = 4.0091e-04, PNorm = 68.6195, GNorm = 0.6394, lr_0 = 8.6198e-04
Validation prc-auc = 0.326389
Epoch 95
Validation prc-auc = 0.348834
Epoch 96
Validation prc-auc = 0.366250
Epoch 97
Validation prc-auc = 0.392110
Epoch 98
Validation prc-auc = 0.375506
Epoch 99
Loss = 1.2029e-03, PNorm = 68.7070, GNorm = 1.5601, lr_0 = 8.5461e-04
Validation prc-auc = 0.422094
Epoch 100
Validation prc-auc = 0.414289
Epoch 101
Validation prc-auc = 0.409515
Epoch 102
Validation prc-auc = 0.380271
Epoch 103
Validation prc-auc = 0.363094
Epoch 104
Loss = 5.5776e-04, PNorm = 68.7855, GNorm = 1.0667, lr_0 = 8.4730e-04
Validation prc-auc = 0.364572
Epoch 105
Validation prc-auc = 0.383391
Epoch 106
Validation prc-auc = 0.395794
Epoch 107
Validation prc-auc = 0.417025
Epoch 108
Validation prc-auc = 0.417415
Epoch 109
Loss = 3.9318e-04, PNorm = 68.8889, GNorm = 0.5353, lr_0 = 8.4006e-04
Validation prc-auc = 0.423294
Epoch 110
Validation prc-auc = 0.407999
Epoch 111
Validation prc-auc = 0.405486
Epoch 112
Validation prc-auc = 0.389085
Epoch 113
Validation prc-auc = 0.398134
Epoch 114
Loss = 6.5463e-04, PNorm = 68.9487, GNorm = 1.7691, lr_0 = 8.3288e-04
Validation prc-auc = 0.382530
Epoch 115
Validation prc-auc = 0.396728
Epoch 116
Validation prc-auc = 0.401089
Epoch 117
Validation prc-auc = 0.412066
Epoch 118
Validation prc-auc = 0.410428
Epoch 119
Loss = 1.5836e-03, PNorm = 69.0320, GNorm = 2.1849, lr_0 = 8.2576e-04
Validation prc-auc = 0.393503
Epoch 120
Validation prc-auc = 0.396847
Epoch 121
Validation prc-auc = 0.393572
Epoch 122
Validation prc-auc = 0.380673
Epoch 123
Validation prc-auc = 0.383298
Epoch 124
Loss = 5.2953e-04, PNorm = 69.0875, GNorm = 1.0398, lr_0 = 8.1870e-04
Validation prc-auc = 0.395001
Epoch 125
Validation prc-auc = 0.387466
Epoch 126
Validation prc-auc = 0.387190
Epoch 127
Validation prc-auc = 0.388580
Epoch 128
Validation prc-auc = 0.408502
Epoch 129
Loss = 1.1344e-04, PNorm = 69.1583, GNorm = 0.2590, lr_0 = 8.1170e-04
Validation prc-auc = 0.407548
Epoch 130
Validation prc-auc = 0.400131
Epoch 131
Validation prc-auc = 0.404819
Epoch 132
Validation prc-auc = 0.418332
Epoch 133
Validation prc-auc = 0.412098
Epoch 134
Loss = 9.5439e-05, PNorm = 69.2128, GNorm = 0.0910, lr_0 = 8.0476e-04
Validation prc-auc = 0.405850
Epoch 135
Validation prc-auc = 0.412293
Epoch 136
Validation prc-auc = 0.385976
Epoch 137
Validation prc-auc = 0.388799
Epoch 138
Validation prc-auc = 0.389300
Epoch 139
Loss = 1.6182e-04, PNorm = 69.2745, GNorm = 1.1281, lr_0 = 7.9788e-04
Validation prc-auc = 0.420078
Epoch 140
Validation prc-auc = 0.421097
Epoch 141
Validation prc-auc = 0.415068
Epoch 142
Validation prc-auc = 0.392616
Epoch 143
Validation prc-auc = 0.363828
Epoch 144
Loss = 8.1901e-05, PNorm = 69.3192, GNorm = 0.0943, lr_0 = 7.9106e-04
Validation prc-auc = 0.345197
Epoch 145
Validation prc-auc = 0.243361
Epoch 146
Validation prc-auc = 0.229676
Epoch 147
Validation prc-auc = 0.354252
Epoch 148
Validation prc-auc = 0.391731
Epoch 149
Loss = 4.2294e-04, PNorm = 69.3643, GNorm = 1.0090, lr_0 = 7.8430e-04
Validation prc-auc = 0.434009
Best score so far is 0.434009 at epoch 149 
Epoch 150
Validation prc-auc = 0.435138
Best score so far is 0.435138 at epoch 150 
Epoch 151
Validation prc-auc = 0.398878
Epoch 152
Validation prc-auc = 0.373007
Epoch 153
Validation prc-auc = 0.362677
Epoch 154
Loss = 3.9616e-05, PNorm = 69.5005, GNorm = 0.0548, lr_0 = 7.7759e-04
Validation prc-auc = 0.339400
Epoch 155
Validation prc-auc = 0.374014
Epoch 156
Validation prc-auc = 0.401272
Epoch 157
Validation prc-auc = 0.432095
Epoch 158
Validation prc-auc = 0.457933
Best score so far is 0.457933 at epoch 158 
Epoch 159
Loss = 2.2208e-04, PNorm = 69.5982, GNorm = 0.6141, lr_0 = 7.7095e-04
Validation prc-auc = 0.457231
Epoch 160
Validation prc-auc = 0.459002
Best score so far is 0.459002 at epoch 160 
Epoch 161
Validation prc-auc = 0.435097
Epoch 162
Validation prc-auc = 0.426581
Epoch 163
Validation prc-auc = 0.390051
Epoch 164
Loss = 9.5034e-05, PNorm = 69.6726, GNorm = 0.2841, lr_0 = 7.6436e-04
Validation prc-auc = 0.372343
Epoch 165
Validation prc-auc = 0.356812
Epoch 166
Validation prc-auc = 0.354596
Epoch 167
Validation prc-auc = 0.354642
Epoch 168
Validation prc-auc = 0.351875
Epoch 169
Loss = 2.0671e-04, PNorm = 69.7238, GNorm = 0.1466, lr_0 = 7.5782e-04
Validation prc-auc = 0.352224
Epoch 170
Validation prc-auc = 0.362880
Epoch 171
Validation prc-auc = 0.364591
Epoch 172
Validation prc-auc = 0.371604
Epoch 173
Validation prc-auc = 0.397754
Epoch 174
Loss = 6.9495e-05, PNorm = 69.7659, GNorm = 0.1663, lr_0 = 7.5134e-04
Validation prc-auc = 0.397765
Epoch 175
Validation prc-auc = 0.401890
Epoch 176
Validation prc-auc = 0.403911
Epoch 177
Validation prc-auc = 0.378353
Epoch 178
Validation prc-auc = 0.370373
Epoch 179
Loss = 2.4117e-04, PNorm = 69.8106, GNorm = 0.7701, lr_0 = 7.4492e-04
Validation prc-auc = 0.369110
Epoch 180
Validation prc-auc = 0.378111
Epoch 181
Validation prc-auc = 0.417868
Epoch 182
Validation prc-auc = 0.399899
Epoch 183
Validation prc-auc = 0.437596
Epoch 184
Loss = 1.2725e-04, PNorm = 69.8609, GNorm = 0.3149, lr_0 = 7.3855e-04
Validation prc-auc = 0.444346
Epoch 185
Validation prc-auc = 0.440594
Epoch 186
Validation prc-auc = 0.432174
Epoch 187
Validation prc-auc = 0.435181
Epoch 188
Validation prc-auc = 0.426698
Epoch 189
Loss = 6.0160e-04, PNorm = 69.9129, GNorm = 1.1941, lr_0 = 7.3224e-04
Validation prc-auc = 0.418538
Epoch 190
Validation prc-auc = 0.394254
Epoch 191
Validation prc-auc = 0.383166
Epoch 192
Validation prc-auc = 0.355454
Epoch 193
Validation prc-auc = 0.350000
Epoch 194
Loss = 6.6889e-05, PNorm = 69.9688, GNorm = 0.2409, lr_0 = 7.2598e-04
Validation prc-auc = 0.351802
Epoch 195
Validation prc-auc = 0.348445
Epoch 196
Validation prc-auc = 0.367797
Epoch 197
Validation prc-auc = 0.392749
Epoch 198
Validation prc-auc = 0.399257
Epoch 199
Loss = 1.1738e-03, PNorm = 70.0234, GNorm = 1.7010, lr_0 = 7.1977e-04
Validation prc-auc = 0.396723
Epoch 200
Validation prc-auc = 0.384583
Epoch 201
Validation prc-auc = 0.357135
Epoch 202
Validation prc-auc = 0.346268
Epoch 203
Validation prc-auc = 0.321916
Epoch 204
Loss = 1.4613e-04, PNorm = 70.0645, GNorm = 0.4615, lr_0 = 7.1362e-04
Validation prc-auc = 0.315241
Epoch 205
Validation prc-auc = 0.359886
Epoch 206
Validation prc-auc = 0.369591
Epoch 207
Validation prc-auc = 0.375038
Epoch 208
Validation prc-auc = 0.385212
Epoch 209
Loss = 4.4293e-05, PNorm = 70.0920, GNorm = 0.1275, lr_0 = 7.0752e-04
Validation prc-auc = 0.382193
Epoch 210
Validation prc-auc = 0.404728
Epoch 211
Validation prc-auc = 0.452168
Epoch 212
Validation prc-auc = 0.457662
Epoch 213
Validation prc-auc = 0.456972
Epoch 214
Loss = 4.1744e-04, PNorm = 70.1248, GNorm = 1.7013, lr_0 = 7.0147e-04
Validation prc-auc = 0.456848
Epoch 215
Validation prc-auc = 0.434673
Epoch 216
Validation prc-auc = 0.405973
Epoch 217
Validation prc-auc = 0.407273
Epoch 218
Validation prc-auc = 0.417229
Epoch 219
Loss = 3.1385e-05, PNorm = 70.1623, GNorm = 0.1417, lr_0 = 6.9548e-04
Validation prc-auc = 0.409904
Epoch 220
Validation prc-auc = 0.389301
Epoch 221
Validation prc-auc = 0.400216
Epoch 222
Validation prc-auc = 0.399124
Epoch 223
Validation prc-auc = 0.391019
Epoch 224
Loss = 7.3179e-05, PNorm = 70.1942, GNorm = 0.0435, lr_0 = 6.8953e-04
Validation prc-auc = 0.392096
Epoch 225
Validation prc-auc = 0.389737
Epoch 226
Validation prc-auc = 0.376502
Epoch 227
Validation prc-auc = 0.360868
Epoch 228
Validation prc-auc = 0.358303
Epoch 229
Loss = 1.4429e-04, PNorm = 70.2170, GNorm = 0.6865, lr_0 = 6.8364e-04
Validation prc-auc = 0.390242
Epoch 230
Validation prc-auc = 0.398067
Epoch 231
Validation prc-auc = 0.401580
Epoch 232
Validation prc-auc = 0.416383
Epoch 233
Validation prc-auc = 0.421761
Epoch 234
Loss = 8.7797e-05, PNorm = 70.2400, GNorm = 0.0076, lr_0 = 6.7779e-04
Validation prc-auc = 0.410764
Epoch 235
Validation prc-auc = 0.416128
Epoch 236
Validation prc-auc = 0.427982
Epoch 237
Validation prc-auc = 0.430169
Epoch 238
Validation prc-auc = 0.415723
Epoch 239
Loss = 1.6393e-04, PNorm = 70.2877, GNorm = 0.7569, lr_0 = 6.7200e-04
Validation prc-auc = 0.407376
Epoch 240
Validation prc-auc = 0.403046
Epoch 241
Validation prc-auc = 0.402789
Epoch 242
Validation prc-auc = 0.413417
Epoch 243
Validation prc-auc = 0.419249
Epoch 244
Loss = 2.6142e-04, PNorm = 70.3230, GNorm = 1.5349, lr_0 = 6.6625e-04
Validation prc-auc = 0.416634
Epoch 245
Validation prc-auc = 0.435206
Epoch 246
Validation prc-auc = 0.430787
Epoch 247
Validation prc-auc = 0.421910
Epoch 248
Validation prc-auc = 0.421468
Epoch 249
Loss = 3.5787e-04, PNorm = 70.3645, GNorm = 0.1044, lr_0 = 6.6056e-04
Validation prc-auc = 0.419487
Epoch 250
Validation prc-auc = 0.422210
Epoch 251
Validation prc-auc = 0.423157
Epoch 252
Validation prc-auc = 0.420648
Epoch 253
Validation prc-auc = 0.407486
Epoch 254
Loss = 2.6612e-05, PNorm = 70.4469, GNorm = 0.0610, lr_0 = 6.5491e-04
Validation prc-auc = 0.389332
Epoch 255
Validation prc-auc = 0.382866
Epoch 256
Validation prc-auc = 0.370431
Epoch 257
Validation prc-auc = 0.364662
Epoch 258
Validation prc-auc = 0.349795
Epoch 259
Loss = 6.7332e-04, PNorm = 70.5032, GNorm = 1.7812, lr_0 = 6.4931e-04
Validation prc-auc = 0.368972
Epoch 260
Validation prc-auc = 0.387957
Epoch 261
Validation prc-auc = 0.418292
Epoch 262
Validation prc-auc = 0.424818
Epoch 263
Validation prc-auc = 0.413119
Epoch 264
Loss = 4.8879e-05, PNorm = 70.5124, GNorm = 0.1680, lr_0 = 6.4376e-04
Validation prc-auc = 0.413287
Epoch 265
Validation prc-auc = 0.411486
Epoch 266
Validation prc-auc = 0.406821
Epoch 267
Validation prc-auc = 0.386396
Epoch 268
Validation prc-auc = 0.378188
Epoch 269
Loss = 1.8948e-05, PNorm = 70.5376, GNorm = 0.0503, lr_0 = 6.3826e-04
Validation prc-auc = 0.376947
Epoch 270
Validation prc-auc = 0.371824
Epoch 271
Validation prc-auc = 0.370747
Epoch 272
Validation prc-auc = 0.370717
Epoch 273
Validation prc-auc = 0.366320
Epoch 274
Loss = 8.9138e-06, PNorm = 70.5573, GNorm = 0.0146, lr_0 = 6.3280e-04
Validation prc-auc = 0.373062
Epoch 275
Validation prc-auc = 0.371364
Epoch 276
Validation prc-auc = 0.373646
Epoch 277
Validation prc-auc = 0.368672
Epoch 278
Validation prc-auc = 0.359711
Epoch 279
Loss = 1.4621e-04, PNorm = 70.5751, GNorm = 0.2895, lr_0 = 6.2739e-04
Validation prc-auc = 0.365074
Epoch 280
Validation prc-auc = 0.371056
Epoch 281
Validation prc-auc = 0.382922
Epoch 282
Validation prc-auc = 0.385173
Epoch 283
Validation prc-auc = 0.396080
Epoch 284
Loss = 6.3536e-05, PNorm = 70.6096, GNorm = 0.2282, lr_0 = 6.2203e-04
Validation prc-auc = 0.394952
Epoch 285
Validation prc-auc = 0.403019
Epoch 286
Validation prc-auc = 0.397475
Epoch 287
Validation prc-auc = 0.391337
Epoch 288
Validation prc-auc = 0.387426
Epoch 289
Loss = 1.0712e-05, PNorm = 70.6352, GNorm = 0.0158, lr_0 = 6.1671e-04
Validation prc-auc = 0.384479
Epoch 290
Validation prc-auc = 0.387610
Epoch 291
Validation prc-auc = 0.387600
Epoch 292
Validation prc-auc = 0.391408
Epoch 293
Validation prc-auc = 0.391408
Epoch 294
Loss = 4.9978e-05, PNorm = 70.6551, GNorm = 0.0817, lr_0 = 6.1144e-04
Validation prc-auc = 0.394991
Epoch 295
Validation prc-auc = 0.393786
Epoch 296
Validation prc-auc = 0.394154
Epoch 297
Validation prc-auc = 0.393957
Epoch 298
Validation prc-auc = 0.392696
Epoch 299
Loss = 4.1585e-05, PNorm = 70.6982, GNorm = 0.1519, lr_0 = 6.0621e-04
Validation prc-auc = 0.402660
Model 0 best validation prc-auc = 0.459002 on epoch 160
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.332899
Ensemble test prc-auc = 0.332899
Fold 9
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_prc-auc.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1000,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1000,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'prc-auc',
 'minimize_score': False,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_prc-auc/fold_9',
 'save_smiles_splits': False,
 'seed': 9,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_prc-auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_prc-auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1000, bias=False)
      (W_h): Linear(in_features=1000, out_features=1000, bias=False)
      (W_o): Linear(in_features=1133, out_features=1000, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=1300, out_features=1000, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1000, out_features=1000, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1000, out_features=1, bias=True)
  )
)
Number of parameters = 4,584,001
Moving model to cuda
Epoch 0
Validation prc-auc = 0.450037
Best score so far is 0.450037 at epoch 0 
Epoch 1
Validation prc-auc = 0.370305
Epoch 2
Validation prc-auc = 0.388645
Epoch 3
Validation prc-auc = 0.378389
Epoch 4
Loss = 8.9579e-03, PNorm = 66.7716, GNorm = 2.2452, lr_0 = 6.5000e-04
Validation prc-auc = 0.362785
Epoch 5
Validation prc-auc = 0.316462
Epoch 6
Validation prc-auc = 0.317289
Epoch 7
Validation prc-auc = 0.360047
Epoch 8
Validation prc-auc = 0.467060
Best score so far is 0.467060 at epoch 8 
Epoch 9
Loss = 4.9848e-03, PNorm = 66.9264, GNorm = 1.0384, lr_0 = 9.9743e-04
Validation prc-auc = 0.501821
Best score so far is 0.501821 at epoch 9 
Epoch 10
Validation prc-auc = 0.487176
Epoch 11
Validation prc-auc = 0.478058
Epoch 12
Validation prc-auc = 0.405461
Epoch 13
Validation prc-auc = 0.283481
Epoch 14
Loss = 4.6293e-03, PNorm = 67.1088, GNorm = 3.7418, lr_0 = 9.8890e-04
Validation prc-auc = 0.205497
Epoch 15
Validation prc-auc = 0.362407
Epoch 16
Validation prc-auc = 0.449887
Epoch 17
Validation prc-auc = 0.409010
Epoch 18
Validation prc-auc = 0.420641
Epoch 19
Loss = 3.6182e-03, PNorm = 67.2765, GNorm = 2.1490, lr_0 = 9.8045e-04
Validation prc-auc = 0.346703
Epoch 20
Validation prc-auc = 0.324890
Epoch 21
Validation prc-auc = 0.353396
Epoch 22
Validation prc-auc = 0.382732
Epoch 23
Validation prc-auc = 0.386688
Epoch 24
Loss = 2.3883e-03, PNorm = 67.4138, GNorm = 2.3632, lr_0 = 9.7207e-04
Validation prc-auc = 0.415662
Epoch 25
Validation prc-auc = 0.426721
Epoch 26
Validation prc-auc = 0.461391
Epoch 27
Validation prc-auc = 0.470609
Epoch 28
Validation prc-auc = 0.494038
Epoch 29
Loss = 1.4285e-03, PNorm = 67.5272, GNorm = 1.0510, lr_0 = 9.6376e-04
Validation prc-auc = 0.437166
Epoch 30
Validation prc-auc = 0.282110
Epoch 31
Validation prc-auc = 0.249586
Epoch 32
Validation prc-auc = 0.279560
Epoch 33
Validation prc-auc = 0.398783
Epoch 34
Loss = 2.4209e-03, PNorm = 67.6257, GNorm = 1.7783, lr_0 = 9.5552e-04
Validation prc-auc = 0.415586
Epoch 35
Validation prc-auc = 0.414906
Epoch 36
Validation prc-auc = 0.414171
Epoch 37
Validation prc-auc = 0.399775
Epoch 38
Validation prc-auc = 0.361135
Epoch 39
Loss = 1.6924e-03, PNorm = 67.7261, GNorm = 0.8603, lr_0 = 9.4735e-04
Validation prc-auc = 0.253146
Epoch 40
Validation prc-auc = 0.248952
Epoch 41
Validation prc-auc = 0.240747
Epoch 42
Validation prc-auc = 0.268443
Epoch 43
Validation prc-auc = 0.378336
Epoch 44
Loss = 1.0404e-03, PNorm = 67.8215, GNorm = 0.7980, lr_0 = 9.3925e-04
Validation prc-auc = 0.388250
Epoch 45
Validation prc-auc = 0.371332
Epoch 46
Validation prc-auc = 0.377379
Epoch 47
Validation prc-auc = 0.390196
Epoch 48
Validation prc-auc = 0.400385
Epoch 49
Loss = 1.5837e-03, PNorm = 67.9205, GNorm = 1.4377, lr_0 = 9.3122e-04
Validation prc-auc = 0.389707
Epoch 50
Validation prc-auc = 0.380465
Epoch 51
Validation prc-auc = 0.365569
Epoch 52
Validation prc-auc = 0.343961
Epoch 53
Validation prc-auc = 0.361172
Epoch 54
Loss = 1.4204e-03, PNorm = 68.0120, GNorm = 2.0245, lr_0 = 9.2326e-04
Validation prc-auc = 0.337816
Epoch 55
Validation prc-auc = 0.363756
Epoch 56
Validation prc-auc = 0.390079
Epoch 57
Validation prc-auc = 0.415987
Epoch 58
Validation prc-auc = 0.417003
Epoch 59
Loss = 6.6322e-04, PNorm = 68.1070, GNorm = 0.2050, lr_0 = 9.1537e-04
Validation prc-auc = 0.412154
Epoch 60
Validation prc-auc = 0.392791
Epoch 61
Validation prc-auc = 0.360464
Epoch 62
Validation prc-auc = 0.355512
Epoch 63
Validation prc-auc = 0.371078
Epoch 64
Loss = 1.0457e-03, PNorm = 68.2006, GNorm = 0.8924, lr_0 = 9.0754e-04
Validation prc-auc = 0.387292
Epoch 65
Validation prc-auc = 0.390836
Epoch 66
Validation prc-auc = 0.386355
Epoch 67
Validation prc-auc = 0.355157
Epoch 68
Validation prc-auc = 0.320615
Epoch 69
Loss = 4.5177e-04, PNorm = 68.2862, GNorm = 0.3958, lr_0 = 8.9978e-04
Validation prc-auc = 0.280934
Epoch 70
Validation prc-auc = 0.285101
Epoch 71
Validation prc-auc = 0.298915
Epoch 72
Validation prc-auc = 0.393805
Epoch 73
Validation prc-auc = 0.394449
Epoch 74
Loss = 2.4077e-04, PNorm = 68.3549, GNorm = 0.2080, lr_0 = 8.9209e-04
Validation prc-auc = 0.416637
Epoch 75
Validation prc-auc = 0.402364
Epoch 76
Validation prc-auc = 0.397900
Epoch 77
Validation prc-auc = 0.386751
Epoch 78
Validation prc-auc = 0.374442
Epoch 79
Loss = 2.4154e-03, PNorm = 68.4241, GNorm = 2.7355, lr_0 = 8.8447e-04
Validation prc-auc = 0.380114
Epoch 80
Validation prc-auc = 0.387017
Epoch 81
Validation prc-auc = 0.383155
Epoch 82
Validation prc-auc = 0.366448
Epoch 83
Validation prc-auc = 0.347813
Epoch 84
Loss = 7.0628e-04, PNorm = 68.4817, GNorm = 0.2407, lr_0 = 8.7691e-04
Validation prc-auc = 0.305652
Epoch 85
Validation prc-auc = 0.339533
Epoch 86
Validation prc-auc = 0.394700
Epoch 87
Validation prc-auc = 0.413091
Epoch 88
Validation prc-auc = 0.421421
Epoch 89
Loss = 7.4661e-04, PNorm = 68.5628, GNorm = 1.7954, lr_0 = 8.6941e-04
Validation prc-auc = 0.422363
Epoch 90
Validation prc-auc = 0.407359
Epoch 91
Validation prc-auc = 0.372122
Epoch 92
Validation prc-auc = 0.361665
Epoch 93
Validation prc-auc = 0.370067
Epoch 94
Loss = 5.0632e-04, PNorm = 68.6427, GNorm = 1.0921, lr_0 = 8.6198e-04
Validation prc-auc = 0.405087
Epoch 95
Validation prc-auc = 0.425877
Epoch 96
Validation prc-auc = 0.427748
Epoch 97
Validation prc-auc = 0.429628
Epoch 98
Validation prc-auc = 0.421362
Epoch 99
Loss = 1.8946e-03, PNorm = 68.7383, GNorm = 2.9622, lr_0 = 8.5461e-04
Validation prc-auc = 0.419851
Epoch 100
Validation prc-auc = 0.412566
Epoch 101
Validation prc-auc = 0.401937
Epoch 102
Validation prc-auc = 0.398018
Epoch 103
Validation prc-auc = 0.356277
Epoch 104
Loss = 8.8744e-05, PNorm = 68.8314, GNorm = 0.2508, lr_0 = 8.4730e-04
Validation prc-auc = 0.350306
Epoch 105
Validation prc-auc = 0.352465
Epoch 106
Validation prc-auc = 0.345386
Epoch 107
Validation prc-auc = 0.357255
Epoch 108
Validation prc-auc = 0.382226
Epoch 109
Loss = 3.4837e-04, PNorm = 68.8963, GNorm = 0.3903, lr_0 = 8.4006e-04
Validation prc-auc = 0.403081
Epoch 110
Validation prc-auc = 0.424053
Epoch 111
Validation prc-auc = 0.421963
Epoch 112
Validation prc-auc = 0.424138
Epoch 113
Validation prc-auc = 0.434068
Epoch 114
Loss = 7.0651e-04, PNorm = 68.9450, GNorm = 1.1043, lr_0 = 8.3288e-04
Validation prc-auc = 0.453837
Epoch 115
Validation prc-auc = 0.446531
Epoch 116
Validation prc-auc = 0.447871
Epoch 117
Validation prc-auc = 0.445324
Epoch 118
Validation prc-auc = 0.431855
Epoch 119
Loss = 5.2400e-04, PNorm = 68.9993, GNorm = 1.6845, lr_0 = 8.2576e-04
Validation prc-auc = 0.403037
Epoch 120
Validation prc-auc = 0.451953
Epoch 121
Validation prc-auc = 0.457940
Epoch 122
Validation prc-auc = 0.450760
Epoch 123
Validation prc-auc = 0.444695
Epoch 124
Loss = 1.0948e-03, PNorm = 69.0552, GNorm = 1.6007, lr_0 = 8.1870e-04
Validation prc-auc = 0.439729
Epoch 125
Validation prc-auc = 0.420354
Epoch 126
Validation prc-auc = 0.394249
Epoch 127
Validation prc-auc = 0.384935
Epoch 128
Validation prc-auc = 0.372492
Epoch 129
Loss = 1.5354e-03, PNorm = 69.1469, GNorm = 2.7961, lr_0 = 8.1170e-04
Validation prc-auc = 0.386628
Epoch 130
Validation prc-auc = 0.432877
Epoch 131
Validation prc-auc = 0.422637
Epoch 132
Validation prc-auc = 0.420004
Epoch 133
Validation prc-auc = 0.426625
Epoch 134
Loss = 6.8034e-05, PNorm = 69.2377, GNorm = 0.1243, lr_0 = 8.0476e-04
Validation prc-auc = 0.424847
Epoch 135
Validation prc-auc = 0.419478
Epoch 136
Validation prc-auc = 0.421315
Epoch 137
Validation prc-auc = 0.418180
Epoch 138
Validation prc-auc = 0.416320
Epoch 139
Loss = 3.4197e-04, PNorm = 69.2925, GNorm = 0.6910, lr_0 = 7.9788e-04
Validation prc-auc = 0.414384
Epoch 140
Validation prc-auc = 0.411573
Epoch 141
Validation prc-auc = 0.417637
Epoch 142
Validation prc-auc = 0.416465
Epoch 143
Validation prc-auc = 0.418943
Epoch 144
Loss = 1.0561e-04, PNorm = 69.3251, GNorm = 0.1287, lr_0 = 7.9106e-04
Validation prc-auc = 0.430208
Epoch 145
Validation prc-auc = 0.429669
Epoch 146
Validation prc-auc = 0.430482
Epoch 147
Validation prc-auc = 0.424737
Epoch 148
Validation prc-auc = 0.414524
Epoch 149
Loss = 5.4861e-05, PNorm = 69.3562, GNorm = 0.0716, lr_0 = 7.8430e-04
Validation prc-auc = 0.404346
Epoch 150
Validation prc-auc = 0.401281
Epoch 151
Validation prc-auc = 0.385598
Epoch 152
Validation prc-auc = 0.375413
Epoch 153
Validation prc-auc = 0.369679
Epoch 154
Loss = 4.4318e-04, PNorm = 69.3884, GNorm = 1.1945, lr_0 = 7.7759e-04
Validation prc-auc = 0.358385
Epoch 155
Validation prc-auc = 0.337252
Epoch 156
Validation prc-auc = 0.349025
Epoch 157
Validation prc-auc = 0.370453
Epoch 158
Validation prc-auc = 0.401653
Epoch 159
Loss = 9.2033e-04, PNorm = 69.4234, GNorm = 2.1035, lr_0 = 7.7095e-04
Validation prc-auc = 0.415435
Epoch 160
Validation prc-auc = 0.418343
Epoch 161
Validation prc-auc = 0.426125
Epoch 162
Validation prc-auc = 0.433050
Epoch 163
Validation prc-auc = 0.397396
Epoch 164
Loss = 5.6901e-05, PNorm = 69.4800, GNorm = 0.1548, lr_0 = 7.6436e-04
Validation prc-auc = 0.377973
Epoch 165
Validation prc-auc = 0.362984
Epoch 166
Validation prc-auc = 0.332042
Epoch 167
Validation prc-auc = 0.274824
Epoch 168
Validation prc-auc = 0.267631
Epoch 169
Loss = 3.6004e-04, PNorm = 69.5405, GNorm = 1.0218, lr_0 = 7.5782e-04
Validation prc-auc = 0.265278
Epoch 170
Validation prc-auc = 0.254682
Epoch 171
Validation prc-auc = 0.254124
Epoch 172
Validation prc-auc = 0.252245
Epoch 173
Validation prc-auc = 0.271996
Epoch 174
Loss = 1.3207e-04, PNorm = 69.6027, GNorm = 0.1995, lr_0 = 7.5134e-04
Validation prc-auc = 0.281123
Epoch 175
Validation prc-auc = 0.337418
Epoch 176
Validation prc-auc = 0.390275
Epoch 177
Validation prc-auc = 0.398201
Epoch 178
Validation prc-auc = 0.404892
Epoch 179
Loss = 3.8388e-04, PNorm = 69.6662, GNorm = 0.6989, lr_0 = 7.4492e-04
Validation prc-auc = 0.410404
Epoch 180
Validation prc-auc = 0.429582
Epoch 181
Validation prc-auc = 0.433564
Epoch 182
Validation prc-auc = 0.435011
Epoch 183
Validation prc-auc = 0.437688
Epoch 184
Loss = 1.7353e-04, PNorm = 69.7190, GNorm = 0.8308, lr_0 = 7.3855e-04
Validation prc-auc = 0.442360
Epoch 185
Validation prc-auc = 0.443739
Epoch 186
Validation prc-auc = 0.446393
Epoch 187
Validation prc-auc = 0.450195
Epoch 188
Validation prc-auc = 0.454734
Epoch 189
Loss = 3.7706e-05, PNorm = 69.7536, GNorm = 0.1135, lr_0 = 7.3224e-04
Validation prc-auc = 0.461595
Epoch 190
Validation prc-auc = 0.462077
Epoch 191
Validation prc-auc = 0.453702
Epoch 192
Validation prc-auc = 0.409160
Epoch 193
Validation prc-auc = 0.432033
Epoch 194
Loss = 7.2508e-05, PNorm = 69.8026, GNorm = 0.0951, lr_0 = 7.2598e-04
Validation prc-auc = 0.405286
Epoch 195
Validation prc-auc = 0.420167
Epoch 196
Validation prc-auc = 0.422930
Epoch 197
Validation prc-auc = 0.419531
Epoch 198
Validation prc-auc = 0.421618
Epoch 199
Loss = 6.8286e-05, PNorm = 69.8435, GNorm = 0.2889, lr_0 = 7.1977e-04
Validation prc-auc = 0.421169
Epoch 200
Validation prc-auc = 0.419799
Epoch 201
Validation prc-auc = 0.420684
Epoch 202
Validation prc-auc = 0.419284
Epoch 203
Validation prc-auc = 0.404511
Epoch 204
Loss = 1.6236e-05, PNorm = 69.8697, GNorm = 0.0081, lr_0 = 7.1362e-04
Validation prc-auc = 0.408230
Epoch 205
Validation prc-auc = 0.408857
Epoch 206
Validation prc-auc = 0.409290
Epoch 207
Validation prc-auc = 0.409590
Epoch 208
Validation prc-auc = 0.430966
Epoch 209
Loss = 2.1973e-05, PNorm = 69.8889, GNorm = 0.0166, lr_0 = 7.0752e-04
Validation prc-auc = 0.433113
Epoch 210
Validation prc-auc = 0.438445
Epoch 211
Validation prc-auc = 0.438537
Epoch 212
Validation prc-auc = 0.438546
Epoch 213
Validation prc-auc = 0.432444
Epoch 214
Loss = 4.7685e-04, PNorm = 69.9069, GNorm = 0.0051, lr_0 = 7.0147e-04
Validation prc-auc = 0.421633
Epoch 215
Validation prc-auc = 0.396911
Epoch 216
Validation prc-auc = 0.391856
Epoch 217
Validation prc-auc = 0.397947
Epoch 218
Validation prc-auc = 0.433951
Epoch 219
Loss = 2.9458e-05, PNorm = 69.9225, GNorm = 0.1093, lr_0 = 6.9548e-04
Validation prc-auc = 0.426764
Epoch 220
Validation prc-auc = 0.424325
Epoch 221
Validation prc-auc = 0.426169
Epoch 222
Validation prc-auc = 0.425916
Epoch 223
Validation prc-auc = 0.420980
Epoch 224
Loss = 6.6038e-04, PNorm = 69.9646, GNorm = 0.2137, lr_0 = 6.8953e-04
Validation prc-auc = 0.403954
Epoch 225
Validation prc-auc = 0.387854
Epoch 226
Validation prc-auc = 0.366840
Epoch 227
Validation prc-auc = 0.368183
Epoch 228
Validation prc-auc = 0.374743
Epoch 229
Loss = 1.0113e-04, PNorm = 70.0259, GNorm = 0.1563, lr_0 = 6.8364e-04
Validation prc-auc = 0.384578
Epoch 230
Validation prc-auc = 0.391483
Epoch 231
Validation prc-auc = 0.400010
Epoch 232
Validation prc-auc = 0.406709
Epoch 233
Validation prc-auc = 0.406048
Epoch 234
Loss = 5.8649e-05, PNorm = 70.1191, GNorm = 0.1590, lr_0 = 6.7779e-04
Validation prc-auc = 0.414602
Epoch 235
Validation prc-auc = 0.398671
Epoch 236
Validation prc-auc = 0.401706
Epoch 237
Validation prc-auc = 0.388475
Epoch 238
Validation prc-auc = 0.388163
Epoch 239
Loss = 9.3193e-05, PNorm = 70.1973, GNorm = 0.6336, lr_0 = 6.7200e-04
Validation prc-auc = 0.386518
Epoch 240
Validation prc-auc = 0.378697
Epoch 241
Validation prc-auc = 0.425109
Epoch 242
Validation prc-auc = 0.447723
Epoch 243
Validation prc-auc = 0.460412
Epoch 244
Loss = 3.4274e-04, PNorm = 70.2705, GNorm = 1.2590, lr_0 = 6.6625e-04
Validation prc-auc = 0.456616
Epoch 245
Validation prc-auc = 0.462309
Epoch 246
Validation prc-auc = 0.449645
Epoch 247
Validation prc-auc = 0.425161
Epoch 248
Validation prc-auc = 0.446263
Epoch 249
Loss = 5.8900e-04, PNorm = 70.3792, GNorm = 2.0280, lr_0 = 6.6056e-04
Validation prc-auc = 0.435622
Epoch 250
Validation prc-auc = 0.361708
Epoch 251
Validation prc-auc = 0.301851
Epoch 252
Validation prc-auc = 0.295151
Epoch 253
Validation prc-auc = 0.294537
Epoch 254
Loss = 2.2710e-04, PNorm = 70.4853, GNorm = 1.1219, lr_0 = 6.5491e-04
Validation prc-auc = 0.296437
Epoch 255
Validation prc-auc = 0.302004
Epoch 256
Validation prc-auc = 0.306761
Epoch 257
Validation prc-auc = 0.316436
Epoch 258
Validation prc-auc = 0.325328
Epoch 259
Loss = 7.2996e-05, PNorm = 70.5622, GNorm = 0.0252, lr_0 = 6.4931e-04
Validation prc-auc = 0.407617
Epoch 260
Validation prc-auc = 0.411250
Epoch 261
Validation prc-auc = 0.411816
Epoch 262
Validation prc-auc = 0.428511
Epoch 263
Validation prc-auc = 0.427474
Epoch 264
Loss = 1.1104e-05, PNorm = 70.6202, GNorm = 0.0159, lr_0 = 6.4376e-04
Validation prc-auc = 0.433341
Epoch 265
Validation prc-auc = 0.433040
Epoch 266
Validation prc-auc = 0.434271
Epoch 267
Validation prc-auc = 0.432867
Epoch 268
Validation prc-auc = 0.435347
Epoch 269
Loss = 1.4722e-04, PNorm = 70.6537, GNorm = 0.6959, lr_0 = 6.3826e-04
Validation prc-auc = 0.432815
Epoch 270
Validation prc-auc = 0.422499
Epoch 271
Validation prc-auc = 0.414197
Epoch 272
Validation prc-auc = 0.406797
Epoch 273
Validation prc-auc = 0.391320
Epoch 274
Loss = 1.6522e-04, PNorm = 70.6827, GNorm = 0.0395, lr_0 = 6.3280e-04
Validation prc-auc = 0.385378
Epoch 275
Validation prc-auc = 0.358782
Epoch 276
Validation prc-auc = 0.344910
Epoch 277
Validation prc-auc = 0.363770
Epoch 278
Validation prc-auc = 0.384140
Epoch 279
Loss = 9.2693e-05, PNorm = 70.7077, GNorm = 0.0512, lr_0 = 6.2739e-04
Validation prc-auc = 0.398806
Epoch 280
Validation prc-auc = 0.400065
Epoch 281
Validation prc-auc = 0.417458
Epoch 282
Validation prc-auc = 0.431318
Epoch 283
Validation prc-auc = 0.411107
Epoch 284
Loss = 8.1547e-06, PNorm = 70.7289, GNorm = 0.0164, lr_0 = 6.2203e-04
Validation prc-auc = 0.408120
Epoch 285
Validation prc-auc = 0.399704
Epoch 286
Validation prc-auc = 0.394869
Epoch 287
Validation prc-auc = 0.387944
Epoch 288
Validation prc-auc = 0.380417
Epoch 289
Loss = 5.5600e-05, PNorm = 70.7561, GNorm = 0.2954, lr_0 = 6.1671e-04
Validation prc-auc = 0.383104
Epoch 290
Validation prc-auc = 0.381922
Epoch 291
Validation prc-auc = 0.382193
Epoch 292
Validation prc-auc = 0.381528
Epoch 293
Validation prc-auc = 0.384386
Epoch 294
Loss = 1.0401e-04, PNorm = 70.7758, GNorm = 0.0911, lr_0 = 6.1144e-04
Validation prc-auc = 0.390045
Epoch 295
Validation prc-auc = 0.387300
Epoch 296
Validation prc-auc = 0.385809
Epoch 297
Validation prc-auc = 0.384128
Epoch 298
Validation prc-auc = 0.384380
Epoch 299
Loss = 8.9965e-06, PNorm = 70.7942, GNorm = 0.0371, lr_0 = 6.0621e-04
Validation prc-auc = 0.384974
Model 0 best validation prc-auc = 0.501821 on epoch 9
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test prc-auc = 0.574495
Ensemble test prc-auc = 0.574495
10-fold cross validation
Seed 0 ==> test prc-auc = 0.536669
Seed 1 ==> test prc-auc = 0.360351
Seed 2 ==> test prc-auc = 0.458042
Seed 3 ==> test prc-auc = 0.443862
Seed 4 ==> test prc-auc = 0.416158
Seed 5 ==> test prc-auc = 0.320944
Seed 6 ==> test prc-auc = 0.546844
Seed 7 ==> test prc-auc = 0.444717
Seed 8 ==> test prc-auc = 0.332899
Seed 9 ==> test prc-auc = 0.574495
Overall test prc-auc = 0.443498 +/- 0.084645
